File-Server Maintenance on 01-07

We will be taking the following file-servers down shortly for maintenance tomorrow 01-07 at approximately 1 AM PST. If one of the following file-servers is listed when clicking “Account Status” when logged into the panel this will affect you:

lewzealand, julius, wanda

It also might cause slightly elevated load during the down time on the following servers:

achilles, aero, alameda, alondra, archibald, arrow, atlantic, avalon, aviation, babyruth, barry, bazooka, beaudry, bellflower, blowpop, bonbon, bottlecap, boyd, broadway, burton, butler, catalina, centinela, century, cerritos, chex, cognac, compton, cookie, cordial, cupcake, dingdong, doheny, dorito, drumstick, dumdum, edendale, euclid, fairfax, fletcher, foothill, fundip, garth, garvey, gobstopper, goober, good, gower, grand, grappa, gumdrop, haven, heath, hillhurst, hillmont, hoover, hope, hops, hyperion, jefferson, jurupa, lankershim, larchmont, laurel, lemonhead, limbo-spunky4, linq, locust, look, lucas, ludlow, malt, manchester, mayo, melrose, mento, milliken, mills, monroe, nilla, nix, oakdale, ocean, olive, ord, pico, pikespeak, planters, plenty, pocky, poprock, pretzel, price, pringle, razzle, redondo, ritz, rockyroad, rodeo, rum, sake, saltine, schnapps, selma, skittle, snocap, soto, spree, starburst, sticky, sunset, taffy, tampa, temple, tequila, topanga, vanowen, vermouth, warhead, washington, western, whittier, whopper, wilshire, york

The downtime for each file-server should be in the range of 20-40 minutes during the maintenance.

Comments: 9

Posted In: General Outages

Posted By: Sandon

Aquaman has drowned

It looks like Aquaman is having some problems with its raid array and is currently down. Our admins are working on the issue and more information is to come when it is known.

Update 04:51 AM PST: Aquaman is currently being moved over to completely new hardware. We currently trying to copy what we can from the old machine and restore whatever else from backups. Unfortunately this machine is very new and we don’t have many backups available.

More updates to come soon.

Update 05:20 AM PST: So far about 8 of the ~200 GB of data has been copied to the new server. So far it has not had any errors so things are looking good. At this rate we are looking at around 5-6 hours to transfer the rest of the data assuming everything goes well. More updates to come later.

Update 08:00 AM PST: About 50% of the data has been copied so far.

Comments: 45

Posted In: General Outages

Posted By: Sandon

Hooray! Another Rails upgrade!

Now that everyone is back to the salt mines for the new year, it’s time for a Ruby on Rails upgrade! 2.2.2 has been released, and we’re not seeing any major complaints about it, so it’s time to upgrade the servers!

We always give our users fair warning because upgrading Ruby on Rails always tends to break currently running applications - but everyone can avoid problems if they freeze their rails! Freezing is way easy. All you have to do is login via to your shell account, change directories into your app, and run:

rake rails:freeze:gems

I’ll be building the packages today and testing them, and if they’re not horrible, I’ll be deploying them across our network slowly as the week progresses. Everything should be done by the end of next week at the very latest!

Comments: 3

Posted In: System Changes

Posted By: Terri

Aquaman is sleeping with the fishes

It would appear that aquaman is currently having issues with serving up some content. Customers have been reporting issues with not being able to see JPG, PNG, GIF, CSS and JS files, amongst others. This means that sites on aquaman will appear to be missing images and theming for the time being — but two members of our admin team are working on fixing this. Updates will come when there’s news of what’s going on.

Update - 6:20 PM, PST: In order to help expedite the resolution of this matter, the server will be inaccessible for the next couple hours. We understand that having a site where people can see some of your content is more desirable than not being able to see anything at all — but I’ve been assured that this will help resolve most issues that customers are seeing.

Update - 7:40 PM, PST: I’ve just received word from the admin in charge of this that all files on aquaman should be served correctly now. If further problems persist, please let our support staff know and we’ll do what we can to make sure you are taken care of.

Comments: 105

Posted In: General Outages

Posted By: DreamHost

DreamHost and Panel Sluggish

We are currently experiencing some slowness on the main DreamHost.com site as well as the web panel (where you get things done!) but are currently working on restoring the services to their full and working order! We’ll post updates as they come!

Update 1:33 PM PDT: The panel and DreamHost site should be back up but we’re still experiencing issues with the sign up pages. More to come…

Update 5:25 PM PDT: We’re still seeing a lot of traffic going toward our signup page. This in turn is still causing the web panel, and some other features (like announcement list signups, and signups themselves!) to have spotty access. We’re doing everything we can to alleviate this, and apologize for the problems! We’ll keep you updated!

Comments: 147

Posted In: Control Panel Outages, General Outages

Posted By: anthony

Ruby 1.8.7 Upgrades

We have started the process of upgrading ruby from 1.8.5 to 1.8.7. This new version is required by some recent ruby software and has been requested several times. In our testing we didn’t discover any issues, but we have now heard about problems with some specific software applications such as RadiantCMS. If you are suddenly having problems with a ruby-based website it may be due to this upgrade and you should check for any updates to the software you are using.

Comments: 9

Posted In: System Changes

Posted By: Dallas

Legs server being moved to new hardware.

In light of the recent issues with this server the admin team has decided it would be best to rebuild it on new hardware. There was a problem with the machine (but not the drives) so it either had ram or bad raid controller and rather than have our customers suffer while we pin it down we’re getting everyone onto a working server. This should be completed within the next hour.

Update: Legs back up!
My apologies for the slow update but it appears that the new improved legs has been up for the last 45 minutes now and should provide greatly improved stability!

Comments: 37

Posted In: General Outages

Posted By: mir

Frisky issues (Sorry but no cute title)

We had an networking issues with the file server (slapper) connection within this cluster, this has been corrected but unfortunately caused some issues with apache services. Our admins have also corrected this and any sites that had experienced downtime due to this should now have their sites coming up normally, if your are still having issues please contact support.

Comments: 6

Posted In: General Outages

Posted By: Javier

yoshi having issues again

We thought that the problems with yoshi were through, but it started acting up again earlier today and required a couple of reboots. After the last reboot it didn’t come back up at all. We moved it to new hardware twice now, so we’ve ruled out the possibility of it being a hardware issue. Our admins are in the datacenter and are working on getting this server back up as soon as they can. Please check back here for updates, as we’ll post them as sson as we have them. We’re sorry for the continued issues with this server, but hopefully we’ll be able to find the cause now.

Comments: 16

Posted In: General Outages

Posted By: danwy

Rack Consolidation Moves

We are consolidating several racks to free up space for new hardware (yay!), consequentially there will be some downtime for the servers involved (boo!). Here’s the list of servers that will be affected:

psmysql1871 psmysql1881 psmysql2460 psmysql2461 psmysql2623 psmysql2624 psmysql2648 psmysql2653 psmysql2660 psmysql2839 psmysql2861 psmysql2865 psmysql2882 psmysql2905 psmysql2934 psmysql3050 psmysql3053 psmysql3054 psmysql3131 psmysql3132 psmysql3159 psmysql3164 psmysql3173 psmysql3226 psmysql3229 psmysql3509 hank stormie redbull brisk milk nastia blogdu bpsites fabsilv flawless foobar fuzzyraygun ps3711 ps3853 ps5130 pssunkist stacyfayehan unadvised vanguardps webon zakidesign algrimur apolo bethndn cihan danjacobs mega mystic ps4088 ps4192 ps4316 ps5119 ps5277 ps5585 ps5965 qikserv001 seventeendots usher wesleybrasil zinkweb alignak ansible arif astore davros digitmidia freewebsites hutchtest protium rawkins tajvps technomatt trashmenagerie appletini calvin fastlane frigate judi krillsc raoul skif albasoun delphae dymweb grafia isnmweb keeeeez sonsuztasarim vejaisso brainsquirt carlosrobles cre8tivegroup filter gannon gravatt igordon izzy lorddjinn nhym pacifico ps5597 ps5902 quagmire reserver sharpie snidely artica binamedia esc ffxiii frances gordini grosbouddha idmo ims intenselighting levkov lion permie pickles sai titania andrewslaterpresents basketmen hpalumni ps3948 ps5861 ps6244 scary webapp erremedia grassroots lerftacular lingolook rocknroll sharktopus skywalker storyhole threesixty goodcow mab pollos ps1 ps4593 ps5661 ps6145 tripter trux webcoot adverstore fanboy loronix nolaflash opiniuj pocapower rightstuff rojax adhoc alpine burning cfinc fishbulb kitchensink maozveg ps4097 ps4158 ps4452 ps4800 quixotic

The majority of the servers listed are Private Servers so there are many of them on a single physical machine. Each physical machine will be down for 5-10 minutes, there are 25 machines in all so we are looking at a window of 2ish/3ish hours. Sorry for any inconveniences!

Update: All machines have survived the move! If you are still experiencing troubles please contact support.

Comments: 15

Posted In: General Outages

Posted By: Kyle