ModernChivalry.org

Main => The Round Table => Topic started by: Sir Edward on 2012-05-01, 15:03:32

Title: Server upgrade
Post by: Sir Edward on 2012-05-01, 15:03:32

Just a head's up that there may be some downtime for the forum in the near future. I haven't chosen when yet, but I'm going to upgrade the linux version on the server to something more modern. When I do this, there will be an outage for all of my websites, including the forum.

In the short-term, I'm going to do a reboot so I can mount the new OS and start doing some configuration quietly in the background. I hate to lose my nearly 3-year uptime on the server, but alas reboots have to happen. :)

This will be the first time I've pulled the rug out from under the entire server since I started it, so I have a lot of little things to check and verify. I'm expecting to have lots of unanticipated problems. I have to copy over the database, webserver configuration, user accounts, back-end scripts, configure a different email server, and so on.

So anyway, just a head's up. If you try to access the forum and can't get in, I'm probably reconfiguring things. I'm not sure if I'll do this during the week or on the weekend.
Title: Re: Server upgrade
Post by: Joshua Santana on 2012-05-01, 15:14:27
Thanks for the warning Sir Edward!  Do let us know via Facebook when the Forum will be back in business!  ;)
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-01, 15:27:27

No worries, when I have the actual outage, my plan is to have it down for no more than a few hours. Depending on your schedule, you may never notice. :)
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-01, 15:31:13

OK, my preliminary reboot is done. So much for my 916 days of uptime (2.5 years). lol. :)

Like I said, I don't know when I'll do the main upgrade, it depends on when I have a few contiguous hours to work on it.
Title: Re: Server upgrade
Post by: Joshua Santana on 2012-05-01, 18:57:21
Thanks for the update Sir Edward, do keep us posted!
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-03, 22:03:14

OK, so here's the plan. I'm hoping to dive into this Saturday morning, with the whole day ahead of me in case things blow up in my face. The forum will definitely be down for some length of time, or even be intermittently visible.

If things go horribly awry, I'll put it all back the way it was and take another stab at it some other time.

If things go much more smoothly than I expect, it'll all be back up and humming along well before lunch.

In reality, it'll probably be somewhere in between, with the big pieces up and running but little details elsewhere that are still broken... meaning, the forum will be up, but some of my back-end stuff with email and DNS and backups and the like might still be hosed. I can deal with cleaning that stuff up later if it comes to that, since the primary work on the server is the web services and the database that runs under them.

It's not often that I rip out the operating system and replace it. I expect much cussing and hair-pulling to ensue, even though I have it pretty well planned out. :)
Title: Re: Server upgrade
Post by: Sir James A on 2012-05-03, 23:39:50
My sympathies in advance, having done a few server upgrades in the past. :)
Title: Re: Server upgrade
Post by: Sir Wolf on 2012-05-04, 00:15:56
good thing i gotta work sat morning. dont wanna go crazy when its down hahaha
Title: Re: Server upgrade
Post by: Joshua Santana on 2012-05-04, 15:05:35
Quote
good thing i gotta work sat morning. dont wanna go crazy when its down hahaha

Hear! Hear!  ;D
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-05, 14:17:48
So far so good. I will probably be fixing small issues for days. But the bigger pieces (the forum primarily) look to be working.

Lots of little headaches along the way of course. The OS starts out really stripped down, so I didn't even have compilers or downloaders for the packages I needed, except for "yum". But little by little, I got things in place. It's been about 2.5 hours so far now.

More to do, but now I can take a deep breath and relax a little. :)
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-05, 22:08:22

Over the course of the day I've managed to fix a lot of the little outstanding details. I'll probably find more as time goes on, and I know I've neglected a couple (graphing and analysis type stuff). But email is working again, got all the little services like FTP, POP, rsync, and so on working, verified a bunch of the startup and scheduled tasks still do what they're supposed to, and so on.

It all went pretty smoothly, except for a couple of brain-farts on my part. All I can say is THANK GOODNESS I know what I'm doing, or this could have been a nightmare. :)
Title: Re: Server upgrade
Post by: Sir Wolf on 2012-05-06, 00:31:41
and we thank you sir!
Title: Re: Server upgrade
Post by: Sir James A on 2012-05-06, 01:53:52
and we thank you sir!

Indeed! Any server upgrade that doesn't involve a screw driver mounted in a vise, with a lawn mower blade on it, is a successful upgrade!
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-26, 02:46:15
Well, I feel stupid. After doing the upgrade, apparently I didn't finish setting up email aliases, so the forum was unable to email me. It's been emailing everyone else, just not me. Figures. :)
Title: Re: Server upgrade
Post by: Ian on 2012-05-26, 14:18:18
Did you make it so the forum content is on a secure connection now?
Title: Re: Server upgrade
Post by: Sir Ulrich on 2012-05-26, 22:35:49
(http://img36.imageshack.us/img36/5348/glitchthatkeepspoppingu.png)
This keeps popping up and it's quite annoying.
Title: Re: Server upgrade
Post by: Sir Wolf on 2012-05-27, 00:42:20
i got a

This Connection is Untrusted





You have asked Firefox to connect
securely to modernchivalry.org, but we can't confirm that your connection is secure.
Normally, when you try to connect securely,
sites will present trusted identification to prove that you are
going to the right place. However, this site's identity can't be verified.



What Should I Do?

If you usually connect to
this site without problems, this error could mean that someone is
trying to impersonate the site, and you shouldn't continue.






Technical Details




I Understand the Risks


dunno why it just randomly came up today
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-27, 01:44:19

Nothing has changed. You should be able to log in with both standard http and https. Is it redirecting you or something?
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-27, 01:48:48
OK, I do see it redirecting to https. It shouldn't be doing that. I'll have to look into it later. For now, ignore.
Title: Re: Server upgrade
Post by: Sir Edward on 2012-05-27, 02:10:48
Arg... so a setting did get changed, and I'm not sure how. Anyway, just change the URL to http, and not https, and you won't see those warnings anymore.

EDIT: I see how it happened. I was changing an unrelated setting, and it automatically set the whole server over to https just because I was logged in that way at the time. Grrr.
Title: Re: Server upgrade
Post by: Sir Gerard de Rodes on 2012-05-27, 21:37:27
Well done and thanks for all your hard work with the forum and site Sir Ed.
G.
Title: Re: Server upgrade
Post by: Joshua Santana on 2012-05-29, 16:53:09
Quote
Well done and thanks for all your hard work with the forum and site Sir Ed.

Thank you Sir Edward for everything and your hard work.
Title: Re: Server upgrade
Post by: Sir Justin on 2012-06-03, 06:39:47

OK, my preliminary reboot is done. So much for my 916 days of uptime (2.5 years). lol. :)

Like I said, I don't know when I'll do the main upgrade, it depends on when I have a few contiguous hours to work on it.

Umm 916 days of continuous uptime, wow nice job! But realize this... one does not simply steal God's computer....
Title: Re: Server upgrade
Post by: Joshua Santana on 2012-06-03, 12:27:48
Quote
mm 916 days of continuous uptime, wow nice job! But realize this... one does not simply steal God's computer....

Indeed.   ;)
Title: Re: Server upgrade
Post by: Sir Edward on 2012-06-03, 14:37:37

lol... :)

What's amusing about that is that linux geeks often put a lot of pride into their uptimes, since a lot of the system management and upgrades can be done without reboots. A reboot is necessary for kernel updates, or of course changing the hardware, but that's about it.

But smarter linux geeks know that the uptimes mean nothing, and going years without a reboot means you haven't tested the startup sequence with whatever updates you've made. And hard drives are mostly likely to fail when powering down and back up. The disk might be going bad and you won't know it until you reboot. :)

But as it turns out, I only got about 16 days of uptime after my upgrades before my hosting service forced a reboot to migrate me to a different physical server, so I'd have rebooted anyway before long. :)
Title: Re: Server upgrade
Post by: Bernarr on 2012-06-10, 10:22:56
Man.. I wish the people I had to deal with realized that simple fact. Whenever we have to reboot a host we get replies to the tickets like "YOU RUINED MY 650 DAY UPTIME!!!!!!"

Yea.. here's the REALLY funny part: I work for your host.
Title: Re: Server upgrade
Post by: Sir Edward on 2012-06-11, 00:24:21
Yea.. here's the REALLY funny part: I work for your host.

You do? Small world. :)
Title: Re: Server upgrade
Post by: Bernarr on 2012-06-11, 02:29:43
Yea, I started here in February of 2011. Never heard of them before a friend told me to apply.
Title: Server upgrade
Post by: Sir Edward on 2012-06-11, 11:42:05
That's cool. I've been a customer for quite a few years, back when the bottom tier plans only had 64 mb of memory. :) I've always been happy with the service.
Title: Re: Server upgrade
Post by: Sir Edward on 2012-06-26, 16:38:34
Yea, I started here in February of 2011. Never heard of them before a friend told me to apply.

BTW, if you don't mind me asking, how large of an operation is it now? When I started as a customer, I think Mr. Aker was a one-man show, and I got the impression there were only a couple dozen host machines at most. I imagine you guys have a robust staff and thousands of hosts now.
Title: Re: Server upgrade
Post by: Bernarr on 2012-06-28, 05:45:46
It's fairly large now. Back on 2006, I left a mid-size host who had close to 9k servers to go work for cPanel, we're almost at that point now I think. Staff has expanded a lot, we just hired some more new people, I think 2 or 3 of those will be starting in July.
Title: Re: Server upgrade
Post by: Sir James A on 2012-06-28, 13:18:27
I checked back over the thread and don't see it ... which host is this? A PM or email is fine. I'm frustrated with my current host. A lot.
Title: Re: Server upgrade
Post by: Sir Edward on 2012-06-28, 14:01:07
I don't know what the etiquette is in talking about one's server host publicly, or what the security implications are, so I've been erring on the side of speaking in generalities. I'll PM you some info.

It's fairly large now. Back on 2006, I left a mid-size host who had close to 9k servers to go work for cPanel, we're almost at that point now I think. Staff has expanded a lot, we just hired some more new people, I think 2 or 3 of those will be starting in July.

Yeah, that's what I was guessing. I've been a customer for 8 years, so I've watched it grow considerably.
Title: Re: Server upgrade
Post by: Bernarr on 2012-07-07, 21:15:22
I don't know what the etiquette is in talking about one's server host publicly, or what the security implications are, so I've been erring on the side of speaking in generalities. I'll PM you some info.

We tend to keep any information we don't want public close to the chest, so you can pretty much feel free to say what you like. You do have a referral code to spread the word, ya know  ;) (You also get $20 in service credit if people sign up using your code and stay active for 90 days)

Also, since James brought up his unhappiness with his current host.. I have a spare cPanel server with plenty of resources just sitting there if anyone wants some space for a site or whatever just let me know!
Title: Re: Server upgrade
Post by: Sir Edward on 2013-05-06, 13:32:51

There will be roughly one hour of downtime on Wednesday, just a head's up:

Wednesday, May 8th, 2013 between 11:00AM UTC and 11:55AM UTC

Our hosting company is doing maintenance and will shut us down briefly.
Title: Re: Server upgrade
Post by: B. Patricius on 2013-05-06, 18:06:03
Thanks for the head's up Sir Edward. 
Good thing being PST, I won't even be awake yet! :D
Title: Re: Server upgrade
Post by: Sir Edward on 2013-05-08, 01:12:28

There will be roughly one hour of downtime on Wednesday, just a head's up:

Wednesday, May 8th, 2013 between 11:00AM UTC and 11:55AM UTC

Our hosting company is doing maintenance and will shut us down briefly.

.... and... now they've canceled it. I'm assuming by "cancel" they mean "postpone", but for now we won't have an outage.
Title: Re: Server upgrade
Post by: Sir Edward on 2014-01-28, 15:00:37
Just in case you noticed the forum was down for about a half-hour this morning, I was resizing the disk partitions to take advantage of the fact that the host doubled my disk space.

Something that I think is hilarious is that they've increased my bandwidth allocation faster than I've needed it. Here it is, the 28th of the month, and it's still showing that I've used up 0% (yes, zero percent, rounded) of my allowed bandwidth:



*  Transfer/mo: 3095 GB
*  Incoming: 1.77 GB
*  Outgoing: 14.1 GB
*  Total: 15.9 GB
You have used 0% of your monthly transfer
Title: Re: Server upgrade
Post by: Sir William on 2014-01-28, 17:55:10
I noticed!  Thought they'd finally found a way to close the door on me.  lol
Title: Re: Server upgrade
Post by: Sir James A on 2014-01-28, 17:57:05
#firstworldproblems :D
Title: Re: Server upgrade
Post by: Sir Nate on 2014-01-29, 16:03:17
#firstworldproblems :D
lol