Www15 Move

[stangz]

Hello Joe,

I am responding to the email that I received this morning at 8:09 am (EST) informing me that my site is being moved at 6:30 pm (EST), which was exactly a 9 hour and 51 minute notice. The email states that "We've been planning this move for a couple of weeks" why was I not informed of this move in the beginning then? It seems quite ridiculous that you have been planning to move me to another server, which is going to affect my site and others, and you have not posted anything to the forums as well as have not sent out an email in the beginning, which should be easy since you have all the email address of everyone on the server.

Also, you state that "We are also asking that you make no changes to your site for a two hour maintenance window beginning when we start the backups (6:30pm) to when we're done - approximately 8:30-9pm ET. Since this is a live server move, we're concerned about forums running on this server." What are you going to do about emails coming into the server? They will obviously change the contents of peoples sites. Are you going to shut down the mail service? If so, how are you going to guarantee that I do not lose any emails? Also, when I change my DNS, there will take some time for it to propagate. What is going to happen to the emails trying to be sent to me during this propagation time?


Also, why can you not give the new server the same IP address as www15 (198.87.86.131). That would make logical sense because then the user would not have to make any changes or have any chance of loss of data (i.e. email).

Also, this is the second time in the past 6 months that you have had to move my site. Is hostpc having a problem keeping their servers up? Is this going to be the trend that I am going to see in the future?

Thank you for your time and I look forward to your response on why there was such poor management on this move!


--Valuable Customer.

[Joe]

Hello "Valuable Customer"

I'll paste the same response that I sent to your helpdesk ticket:

We've been planning this move in the background, to make it as seamless as possible when it was finally implemented.

If both servers had the same IP address, there's no way they could be both on the net on the same time ... one would have to come down while the other is brought online. With this plan, there is no downtime, period. There is no "dns propogation" because we're simply swapping IP addresses for the nameservers. the changeover should be immediate, and seamless - including all mail, etc. In the off chance that it's "not" immediate mail will still be accessible via IP on the old server for the rest of the evening. Swapping servers with the same IP would make this impossible as well.

Mail server TTL's have already been reset to 5 minutes - so we're not expecting much, if any mail to wind up on the old server.

Once the move is completed THEN you update your dns - just as a formality - it doesn't really change, just the entries. You'll see, it's painless and easy. Even if you did have to update DNS and wait for propogation (which you dont) - under the new rules, it's real time DNS reporting. You could register a new domain and have it fully functional on the net in 15 minutes.

As for a "trend" in moving servers, no - it's not a trend. We had some client scripts that were exploited to a point where 2 security analyists felt it better that we get everyone off this server as soon as possible. We've eliminated the problem, and now must move the sites.

Dont get "nervous" about this - you'll see that it's going to be as painless and as casual as that email indicated.... we've been working on perfecting this so there's no "issues" during the move - we expect it to be seamless, and without interruption.

As for the comment on "poor management" - I kind of resent that remark. We've worked hard and meticously to make sure that this move was as streamlined as possible, and tested it a half dozen times to make sure there's no "issues".

Joe

[Joe]

In case anyone is interested or has questions, we're in IRC.HOSTPC.COM port 6668, channel #support

Joe

[Joe]

Backups are taking a bit longer than planned. we're about 75% through the backup, it'll take about an hour for the restore, so we need to enlarge this maintenance window a bit. I'll post more updates and details here and in IRC as they become available.

Joe

[Joe]

Due to a hardware issue with a brand new out of the box server from Dell, we're forced to abandon this move tongiht.

We'll reschedule this again in the near future.

I'm sorry for the inconvenience.

Joe

[muttdog]

Since this topic http://www.hostpc.com/forums/index.php?showtopic=1861 is locked, i post here.

"If both servers had the same IP address, there's no way they could be both on the net on the same time ... one would have to come down while the other is brought online."

Not exactly. You can have two computers (or even more if you like) respond and recieve from a single IP as if they were one computer. Take load balancing servers for example. Or even redundant servers. Or even your home PC behind a router.

You could in fact avoid all problems by setting up a mail queue server. Change the routing dns for mail destine for www15 to go to the mail queue server, which would queue for delivery but not distribute to mailboxes. Then make a image backup of the user data directories on to the new servers, including the mailbox or mbox files. Once that is complete, have the old or dead and the new come live. Then have the mail queue delivered to the new server. Invisible to users, and to users web surfers.


"Even if you did have to update DNS and wait for propogation (which you dont) - under the new rules, it's real time DNS reporting. You could register a new domain and have it fully functional on the net in 15 minutes."
In this case you could have made a server called www16 that is a mirror of www15 data. Then change the www15 entry to point to www16. Then copy and append the user mailbox files to the new mail boxes on www16.


Servers go down all the time, its the ones that go down and back up unnoticed that we never realize there was a problem in the first place.


just a thought.

[stangz]

As for the comment on "poor management" - I kind of resent that remark. We've worked hard and meticously to make sure that this move was as streamlined as possible, and tested it a half dozen times to make sure there's no "issues".

Joe

[thevillageinn]

Originally posted by muttdog@Jan 24 2005, 06:25 PM
You can have two computers (or even more if you like) respond and recieve from a single IP as if they were one computer. Take load balancing servers for example. Or even redundant servers. Or even your home PC behind a router.



Not exactly.

[D9r]

I keep forgetting what server I'm on. 15 sounds familiar, have to go do a ping or something to find out.

... guess I'm #16 - ns16b.privatelabeldns.com, so I can sleep easy tonight. :o

[thevillageinn]

www15 is not a reseller server, where as www16 is. There are some other reseller servers as well as many other regular servers.

[muttdog]

Originally posted by thevillageinn@Jan 25 2005, 01:10 AM
Not exactly.
Quoted post



Why not exactly? Care to explain?

Yes it is tru that you cannot have two of the same IPs on the same routed network submask such as the internet, but that is not to say that there are not two computers that can access and be accessed by the internet, that dont have the same IP.

if you have some networking insight, please share.

[Joe]

A DA license can not exist on two different servers with the same IP address, thats one limitation. We've already got this worked out - and the change will take place after the 3am backups one night this week.

Joe

[George]

Is it possible to have the server move occur at a more convenient hour? Say some odd hour in the morning as opposed to evening, when it's primetime for many of my sites? 3am would be ideal.

And to echoe what the other user wrote before, I too find these amount of "forced moves" to be tremendously annoying. I'm sure you have good reasons for them each time but the fact of the matter is no other host has subjogated me to this many forced moves.

[thevillageinn]

Originally posted by muttdog@Jan 25 2005, 07:22 AM
Why not exactly? Care to explain?

Yes it is tru that you cannot have two of the same IPs on the same routed network submask such as the internet, but that is not to say that there are not two computers that can access and be accessed by the internet, that dont have the same IP.

if you have some networking insight, please share.


I'm not sure whether you and I are saying the same thing, or not. Also, I'm not a networking professional, so here's how I understand it.

This part...

"...but that is not to say that there are not two computers that can access and be accessed by the internet, that dont have the same IP."

is where my confusion lies. Maybe you are thinking about local private IP addresses such as 10.x.x.x or 192.168.x.x? My computer here at home is behind a firewall / router. That box has 2 network cards, one attached to the internet, receiving a unique IP address, and the other card attached to / serving my local network with a 192.168.x.x address.

I think the difference with the DA server setup is that the machine is not on a private network with a private IP behind a router. It's got its own internet IP, which we've already established can't be duplicated.

Did I make sense, and am I on track? (ignoring the part about DA servers requiring separate IPs.)

[muttdog]

Originally posted by thevillageinn@Jan 26 2005, 01:58 AM
is where my confusion lies. Maybe you are thinking about local private IP addresses such as 10.x.x.x or 192.168.x.x? My computer here at home is behind a firewall / router. That box has 2 network cards, one attached to the internet, receiving a unique IP address, and the other card attached to / serving my local network with a 192.168.x.x address.

I think the difference with the DA server setup is that the machine is not on a private network with a private IP behind a router. It's got its own internet IP, which we've already established can't be duplicated.

Did I make sense, and am I on track? (ignoring the part about DA servers requiring separate IPs.)
Quoted post



"local private IP addresses" is a range that is pre-discribed as non-routeable. However, with almost any router, you can set up your own range of IPs to be non-routeable beyond your "routing" point (connection to internet). [this would restrict you from accessing any 'real' computers in that range, but if none the less possible']

regardless of that, the 192.168.x.x type addresses people are fimilar with are still IP addresses, and therefore to the computer its self, it has the same IP as many other computers in the world (wether or not it can get to one another)

again regardless of that 'technicality':
"...but that is not to say that there are not two computers that can access and be accessed by the internet, that dont have the same IP." I was refering to the fact that 'to the outside' world (outside your private network) all other computers on the internet recognize your NAT addressed internal network computers (could be NAT or any other form of proxy or unique routing table, possiblely RIP additionally) as one single IP address. So in your firewall/router example, all computers behind your router are recognized to the outside world as the single IP assigned to your router.

Your router uses it 'method' to route traffic, in this case, the NAT table. however, a 'routing' device can be set up to route based on traffic load, packet request size, incoming request IP range, packet type [tcpip/udp], or even by port (aka mail versus ftp)

So when you push a request to say a popular website, it is not necessarily only finding one single computer with the address 207.46.130.108, it could be multiple computers that 'access and be accessed by the internet' on the same ip. This is typically done to load balance the requests being made across multiple systems. In addition, there could be a "backup server" that is known to the router, in whch case the router would forward packets to him in the event that one primary server become non-responding (aka crash).

[thevillageinn]

what I read is that you are saying load balancing is performed by assigning the same IP to multiple machines on the network (private or public) and then letting routers (I guess) figure out the rest.

That's entirely incorrect, if that's what you are saying. If that's not what you are saying, then I'm sorry I misunderstood you.

[thevillageinn]

back on topic, however, I admit I was a little surprised to see the information regarding the www15 move, but, since this is a full server change and it's being handled from the top level, I am confident that the move will go simply and efficiently. Also, because I don't host any mission critical sites, a little downtime is the least of my worries. To those with e-commerce sites, depending on the size of your business, it might be prudent to consider a dedicated server. If sales are that important, the investment isn't that large.

[stangz]

Originally posted by thevillageinn@Jan 26 2005, 10:29 AM
back on topic, however, I admit I was a little surprised to see the information regarding the www15 move, but, since this is a full server change and it's being handled from the top level, I am confident that the move will go simply and efficiently.
Quoted post


I am not concerned that the move is not going to go well. Hostpc has always strived to make sure that they provide the best possible service that they can. The part that I was upset about was that they just sprung this on me with only a 10 hour notification. And they said they were planning this for over 2 weeks. I have always thought that hostpc was more professional than another other host because they have always shown dedication to courtesy and treating everyone as a very important customer. In light of this, I was very shocked that I was not informed of the move earlier.


Also, because I don't host any mission critical sites, a little downtime is the least of my worries. To those with e-commerce sites, depending on the size of your business, it might be prudent to consider a dedicated server. If sales are that important, the investment isn't that large.

What does a dedicated server have to do with wanting to have reliable uptimes and service?

Also, how much is little downtime. Is 3 hours a year a little downtime? Is 3 hours a month little? Is 3 hours a week? Is 3 hours a day? Is 1 hour a day little? That would amount to the server being up 95% of the time, which would seem pretty good. What if your internet provider kept having these "little" downtimes? I am sure that you would get rather annoyed. Sooner or later, we will hit a point where you think that the downtime is ridiculous. I am not saying that hostpc has a lot of downtime, they are actually known for quite the opposite. But my definition of "little" is probably much different than yours.

Also, any webhost has to know that they are getting into a market which strives on perfection. I don't know of any other market that can attempt to offer 99.9% guaranteed uptime/service. So, if what you offer is perfection, and it doesn't live up to that in the any which way, then you will get complaints.

But, in fact, I was not upset with having to move. I was more concerned with the lack of notification of having to move.

</rant>

[Joe]

I understand your /rant ... but the fact is that I've completed this move on 2 other servers - without ever mentioning it to clients.... there was ZERO downtime, nobody even knew it took place.

Due to the issues on www15, I felt it prudent to notify everyone of the situation before hand - especially since it was scheduled to be completed during "normal hours". The reason for the 7pm change was because IF there was a problem, the datacenter would be staffed sufficiently to assist us with an IP re-allocation, or hardware swap should the need arise. While there's a tech on call at the datacenter 24x7, It's not always prudent to have an overnight "LED watcher" trying to re-allocate IP's.

None the less, I will be performing this upgrade overnight - and again, there will be no dowtime in the transition. Whoever commented that this should be done at a more convenient hour - like 3am, hahaha - convenient for who? :) (just kidding).

Anyway, the move will go forward, and we'll be better off for it. Our current uptime over 18 servers is 99.962% (as of a couple hours ago), and we'll continue to watch that number rise.

Thanks for your continued support

[thevillageinn]

Originally posted by Joe+Jan 27 2005, 05:36 PM--><div class='quotetop'>QUOTE(Joe @ Jan 27 2005, 05:36 PM)</div><div class='quotemain'>I understand your /rant ...
[/b]

I think that's good of you to say, because it took me a while to understand the rant, and I've ranted plenty on prior occasions.

<!--QuoteBegin-stangz@Jan 27 2005, 03:56 PM
What if your internet provider kept having these "little" downtimes? I am sure that you would get rather annoyed. [/quote]

They already do. It is annoying!

[muttdog]

Originally posted by Joe@Jan 27 2005, 08:36 PM
I understand your /rant ... but the fact is that I've completed this move on 2 other servers - without ever mentioning it to clients.... there was ZERO downtime, nobody even knew it took place.
Quoted post



Woah, so now I think i might have been preaching to the choir. i was under the impression that stangz, being on www15, was perturbed because of short notification of server move, and that he would have to migrate data over, without much notice. And that in the event that say he was unavailable to move his stuff over, given the short notice, his site/email might be "down" until his next available time.
Now i am reading and thinking that this might have been a misconception of what is involved during this move? That this talk of redundant/fallover server garbage i been saying is already 'been there done that'? If this is the case, maybe this was a case of ignorance is bliss for a user.


oh and by the way "Microsoft's income is about the same as New York State receives in taxes"- quote from /.

[ShadowLab]

So what's the status on this...?

I tried following the thread, but the combination of my ADHD and the endless ramblings of pissed off clients made it impossible to follow.

I received emails about this last week saying it was going ahead and then there was the last minute cancellation. I haven't received any updates since then and would like to know where we stand.

Thanks, Kevin.

[muttdog]

Originally posted by ShadowLab@Jan 29 2005, 12:52 PM
I tried following the thread, but the combination of my ADHD and the endless ramblings of pissed off clients made it impossible to follow.
Quoted post



Wow, you must have serious ADHD considering there were only two clients that raised an issue, and a total of 21 posts over 5 days, few of which we over 100 words. I cant imagine having ADHD and trying to follow a Fatwallet thread.

[thevillageinn]

LOL...I didn't make it past here...
Originally posted by ShadowLab@Jan 29 2005, 09:52 AM
So what's the status on this...?


The move has been suspended until the hardware issue with the new sever was resolved - which I understand has already happened, but there's been no further word yet.

It's probably already happened.

[ozee]

Any ETA on this yet? I'm wanting to do a fairly major overhaul on one of my domains, but don't want to start until it's moved...

Thanks

[Jeff]

Originally posted by ozee@Feb 5 2005, 09:43 PM
Any ETA on this yet? I'm wanting to do a fairly major overhaul on one of my domains, but don't want to start until it's moved...

Thanks
Quoted post
Unofficially, if I were you, I wouldn't wait for the move. It is looking less likely that a move will be needed.

[ozee]

Originally posted by Jeff+Feb 8 2005, 12:55 PM--><div class='quotetop'>QUOTE(Jeff @ Feb 8 2005, 12:55 PM)</div><div class='quotemain'><!--QuoteBegin-ozee@Feb 5 2005, 09:43 PM
Any ETA on this yet? I'm wanting to do a fairly major overhaul on one of my domains, but don't want to start until it's moved...

Thanks
Quoted post
Unofficially, if I were you, I wouldn't wait for the move. It is looking less likely that a move will be needed.
Quoted post
[/b][/quote]

Can we have Installatron back, then - or should I do a manual install?

[Joe]

Installatron will be re-enabled this evening

[thevillageinn]

are the problems resolving themselves?

are the problems that were forcing a server move related to IMAPs syrupy slow performance on www15 or is that something else entirely...because IMAP seems to be getting slower and slower all the time.

[Jeff]

Originally posted by thevillageinn@Feb 9 2005, 03:05 PM
are the problems resolving themselves?

are the problems that were forcing a server move related to IMAPs syrupy slow performance on www15 or is that something else entirely...because IMAP seems to be getting slower and slower all the time.
Quoted post


IMAP performance is a different issue, discussed here (http://www.hostpc.com/forums/index.php?showtopic=1885&view=getnewpost) that affects all servers.

After extensive analysis, the other issues leading to the move are turning out to be confined to a couple of accounts on the server. The issues with those accounts are being addressed.

If you do observe unusual performance with that server, please do let us know.

Anyone who would like to move off of 15 and onto a newer server is welcome to request that move. FWIW, the server is nearing 1 year in service.

Also, for those who have been eagerly seeking its return, Joe tells me that he intends to re-activate Installatron on 15 tonight! :)

[thevillageinn]

Jeff,
thanks for the response. I had seen that post about IMAP, but I don't understand how it keeps getting worse and worse.

Eitherway, I haven't noticed any issues with my account on www15, other than the IMAP slowness.

[ozee]

Originally posted by Jeff@Feb 9 2005, 03:16 PM
Also, for those who have been eagerly seeking its return, Joe tells me that he intends to re-activate Installatron on 15 tonight! :)
Quoted post



Still waiting... Please??!!

I'm ready to start playing with a major site rework. Since I'm on dialup, I'd much rather let Installatron install some of the stuff I'm wanting to use.

[Joe]

This has been reactived, and updated to the newest releases.

[ozee]

YAY!! Thanks, Joe!