Overview website downtime due to server migration

Discussion in 'Official Announcements' started by 28117_deleted, Apr 14, 2010.

  1. 28117_deleted
    28117_deleted Member
    Dear Community members,

    Shapeways is growing and growing and therefore we need to step over to a bigger server. During this movement we unfortunately expect some down time between the following hours (CET):

    - April 16th 08:00 - 09:00 max 1 hour
    - April 19th 08:00 - 09:00 max 1 hour
    - April 20th 08:00 - 09:00 max 1 hour
    - April 21st 08:00 - 09:00 max 1 hour
    - April 22nd 08:00 - 09:00 max 1 hour

    Sorry for the inconvenience!

    With kind regards,
    The Shapeways Team
     
    Last edited by a moderator: Apr 19, 2010
  2. 11539_deleted
    11539_deleted Member
    Last night we had the first step in our migration, physically relocate the current servers so they are next to the new servers which will allow us to easier move all data (eg model files).

    The move self went pretty smooth, the website was back up and running in about an hour, but unfortunately there was an oversight on the model processing queue. This caused uploaded models to not being processed, and not showing in your 'mydesigns'.

    We did fix that asap, and will have the models uploaded last night, processed later today. So do not despair ... we'll process your model, only this time it takes a bit longer ... Sorry.


    I'll keep you updated in this thread with each step of our migration to the bigger servers.

    Kind regards, Arno

     
    Last edited: Apr 14, 2010
  3. stannum
    stannum Well-Known Member
    For international things, the best is to use UTC, it's more probable to know the local offset than someone's else offset (and you still need the local info anyway). Also, I doubt the times you give are CET, are you sure they aren't CEST? ^_^
     
  4. 11539_deleted
    11539_deleted Member
    Hi all,

    This morning we had another downtime due to the server migration. We had about 35 minutes of down time, from 8:40 till 9:15 (Amsterdam time). So it started a bit late, but we didn't need the full hour.

    Over the weekend we have copied all your model data to the new diskstorage (a DAS) using rsync. This morning we did another rsync (while still online) which took a bit longer as expected (you uploaded a lot of models, hehehe.)
    Then the site was taken offline, another rsync to get the latest models on the new storage, change the mountpoints, and put the site back online.

    We are running the site still on the old servers, but datastorage is already on the new DAS. More news after the next step,

    kind regards, Arno

    -update: That was 6:40 till 7:15 in UTC ;)
     
    Last edited: Apr 19, 2010
  5. 11539_deleted
    11539_deleted Member
    Hi all,

    another update on the server migration.

    Yesterday we managed to do our stuff, without actually needing to shutdown the site (found a workaround, yeah) so no downtimes involved there.

    Today we had planned to change DNS, but all people who can do that for our domain are still in transit (don't you love mother nature and volcano's? ) We want at least one of them on site to prevent 'problems'. ;)

    So, the good news is, no downtime today. The bad news, we'll need a downtime slot next week.

    I'll keep you posted,

    kind regards, Arno
     
  6. 11539_deleted
    11539_deleted Member
    Hi All,

    I have managed to get all pieces for the next migration step come together at 11:45 CET today. (09:45 UTC)

    While my colleagues enjoy an early lunch we'll have the next server migration step. It will be a big step, which also means we'll need about 45min-1 hour.

    Sorry for the inconvenience.

    Kind regards, Arno
     
  7. 11539_deleted
    11539_deleted Member
    Hello,

    another update about the migration.

    We have moved www.shapeways.com to our new server ! :laughing:

    Unfortunately, it didn't go as smooth as we had prepared, also caused a bit by combining the site update for Euro pricing with the move of the site itself to have only one downtime.

    As a result, we started later, and on top of that, it took longer. Sorry for that.

    But let's not forget about the positive things:
    - The site is running on the new server
    - We support Euro's now.

    Also, the main page and galleries are getting a refreshment too. And we fixed various 'small' issues.

    Back to migration:
    - For next week we'll need one more downtime to move the database to it's new server too.
    But then we have fully migrated our server platform to it's newer counterpart which is about 3 times as fast.

    Thanks for your patience,

    Kind regards, Arno

    PS: On some pc's (most noticeably running windows with IE) it will take longer to refresh your DNS entry for shapeways.com. So some of you might end up in visiting the old server. For the time being we have place a redirect to the new IP on that server. You'll be able to see if you were redirected by the old server by the ip-address that shows in the URL.
     
    Last edited: Apr 28, 2010
  8. virtox
    virtox Active Member Moderator
    I use FF and Chrome, and both still revert to the IP address sometimes and sometimes not.

    I flushed all local DNS caches yesterday, which seemed to work for a while. So is it just my ISP with outdated and inconsistent DNS entries ? Or something else amiss ?

    Cheers !


     
  9. pete
    pete Shapeways Employee CEO
    Hi,

    we have changed a few settings and all should now be ok.

    Peter
     
  10. 11539_deleted
    11539_deleted Member
    Hi all,

    The last server migration slot is at the moment scheduled at Monday May 3rd 6:00-7:00 am GMT (8:00 am Amsterdam time).

    It should take about 40 minutes, but to give ourselves some slack, we reserve 1 hour ;)

    I'll update this post on Monday to let you know that all went well.

    Kind regards, Arno
     
  11. 11539_deleted
    11539_deleted Member
    Hi all,

    another update on the migration. This morning, you didn't notice any downtime, coz we didn't have any.

    We started with migrating a part of our test-environment , but encountered some issues. So, today we'll analyze these and solve them. And do another attempt tomorrow morning (May 4th, 6:00 GMT).

    Kind regards, Arno