Jump to content

Shit's Borked


RPM

Recommended Posts

yeah I'm doing some stuff it's just regenerating the cache 60s tops out of whack. refresh and you should be fine. 

in particular I've finally become fed up enough with the bottom alert not being readable on the second line that I'm trying to hunt it down and correct it with some custom css

  • Hook 'Em 1
Link to comment
Share on other sites

3 minutes ago, thunderlounge said:

image.thumb.png.b0448aeb25b0f00837a9ae26b9d40ede.png
 

If you’re playing with css, a kick up on the z-index for the bottom quick nav might be warranted. 
 

Note in the image that the post box covers it up.

 

always a thing or just today? what is that aspect ratio? what device?

Link to comment
Share on other sites

1 hour ago, immamac said:

always a thing or just today? what is that aspect ratio? what device?


Longer than I can remember, ios, safari, either aspect. 
 

I think Mac/FF is that way too, but I’ll double check after bit. 

Edited by thunderlounge
ou sux
Link to comment
Share on other sites

  • 1 month later...
28 minutes ago, immamac said:

Tornado hit data center. Sorry for downtime. Going to have more downtime to fix later

the message board administrator's equivalent to "my dog ate my homework" 

  • Hook 'Em 2
  • Haha 4
Link to comment
Share on other sites

55 minutes ago, immamac said:

Tornado hit data center. Sorry for downtime. Going to have more downtime to fix later

Please report weird shit here but I'm with my family and mobile only. I just spent about an hour on a bench on a phone getting this back up so be patient. 

You should try the cloud. Clouds don’t get hit by tornadoes. 

  • Hook 'Em 2
  • Haha 4
Link to comment
Share on other sites

10 minutes ago, Buzzrock said:

You should try the cloud. Clouds don’t get hit by tornadoes. 

IIRC, he's running surly for almost nothing on some extra capacity he already had in a DC so moving to cloud would be a big jump in TCO for the site. That said, it may be good to find a co location so that the two nodes can't both be knocked offline in the same storm lol

Link to comment
Share on other sites

10 minutes ago, Buzzrock said:

You should try the cloud. Clouds don’t get hit by tornadoes. 

IIRC, he's running surly for almost nothing on some extra capacity he already had in a DC so moving to cloud would be a big jump in TCO for the site. That said, it may be good to find a co location so that the two nodes can't both be knocked offline in the same storm lol

Link to comment
Share on other sites

15 minutes ago, Longhorn_Fan68 said:

Right now when I hit "Submit Reply" it says "Saving" but just spins. If I refresh page my post is there. Might avoid some double posts right now

 

1 minute ago, D3zii said:

Keeps Double posting 

try the above

Link to comment
Share on other sites

5 minutes ago, Captainant said:

IIRC, he's running surly for almost nothing on some extra capacity he already had in a DC so moving to cloud would be a big jump in TCO for the site. That said, it may be good to find a co location so that the two nodes can't both be knocked offline in the same storm lol

Excuse me, sir. Seeing as how the V.P. is such a V.I.P., shouldn't we keep the P.C. on the Q.T.? 'Cause if it leaks to the V.C. he could end up M.I.A., and then we'd all be put on K.P.

  • Hook 'Em 1
Link to comment
Share on other sites

5 minutes ago, Captainant said:

IIRC, he's running surly for almost nothing on some extra capacity he already had in a DC so moving to cloud would be a big jump in TCO for the site. That said, it may be good to find a co location so that the two nodes can't both be knocked offline in the same storm lol

I have a whole DR setup in Chicago (as indicated on Twitter) but I hate flipping DNS to the DR site because it really makes it a huge headache for me when we flip back to the main site. This is the first time in over 8 years the Datacenter has had an issue that impacted the whole thing and got through the 2N + Diesel + UPS power redundancy. This was a catastrophic event and the fact that we only had 4 hours of downtime in a low traffic period is pretty badass. 

Flipping to DR was also my next move as soon as I got home but I can only do so much from a god damned terminal on my phone. 

5 minutes ago, D3zii said:

Keeps Double posting 

Search is causing this. It will fix when all the nodes come back. 

3 minutes ago, Longhorn_Fan68 said:

 

try the above

Yeah don't wait longer than a second or 2 then just refresh or click the last post # and it'll load from there. 

  • Hook 'Em 3
Link to comment
Share on other sites

5 minutes ago, Captainant said:

IIRC, he's running surly for almost nothing on some extra capacity he already had in a DC so moving to cloud would be a big jump in TCO for the site. That said, it may be good to find a co location so that the two nodes can't both be knocked offline in the same storm lol

I have a whole DR setup in Chicago (as indicated on Twitter) but I hate flipping DNS to the DR site because it really makes it a huge headache for me when we flip back to the main site. This is the first time in over 8 years the Datacenter has had an issue that impacted the whole thing and got through the 2N + Diesel + UPS power redundancy. This was a catastrophic event and the fact that we only had 4 hours of downtime in a low traffic period is pretty badass. 

Flipping to DR was also my next move as soon as I got home but I can only do so much from a god damned terminal on my phone. 

5 minutes ago, D3zii said:

Keeps Double posting 

Search is causing this. It will fix when all the nodes come back. 

3 minutes ago, Longhorn_Fan68 said:

 

try the above

Yeah don't wait longer than a second or 2 then just refresh or click the last post # and it'll load from there. 

  • Hook 'Em 2
Link to comment
Share on other sites

1 minute ago, HornOnTheBayou said:

Well that was a rough morning. I got so desperate for something to do, I actually talked to my wife and kids.

I was trying to get shit back up with screaming children doing the dino exhibit at Lady bird wildflower center. 

  • Hook 'Em 2
  • Like 2
Link to comment
Share on other sites

Posted (edited)
1 hour ago, Buzzrock said:

You should try the cloud. Clouds don’t get hit by tornadoes. 

Oh, yeah?

SEI_150845339.jpg?width=1200

 

31 minutes ago, HornOnTheBayou said:

Well that was a rough morning. I got so desperate for something to do, I actually talked to my wife and kids.

I was at Bullneck, Jr's high school graduation.  Gotdamn those people like to talk and speechify.  

Edited by Bullneck
  • Hook 'Em 1
Link to comment
Share on other sites

This is not something that the mods need to fix, only mentioning it in case others have the same issue:

All of my Ignore settings got reset...  I started seeing all these Rex Kramer posts, and when I checked Ignore, all the names were still there, but the settings were to ignore messages (and not ignore posts).  Weird.

Link to comment
Share on other sites

2 minutes ago, Braff Zacklin said:

First pizza, then Whataburger, now Surly. What's left for Chicago to ruin?

Chicago latency from Texas is lowest and geographically far enough to where the same event is unlikely to have effected both. 

Florida has too many hurricanes. North Carolina didn't have any capacity, didn't wanna do Arizona, but could look into it. California or any west region is too high latency for me same with the full on northeast regions. 

Surly is on the backbone, which is one of the reasons we don't have to have a cdn to serve content surly gets to the open web in 1 hop, through the redundant interconnect switch. Surly for users in Texas is actually faster than the majority of CDNs. For users outside of Texas it's likely just as fast. When I travel I test it pretty thoroughly and the latency and bandwidth is totally fine. 

I was only considering locations on backbone and not buried behind 2-3 interconnects to get to there. 

I may set up a data replication service off backbone to make DR trivial, the hard part right now is dealing with data syncing between sites as replication is one way by design right now. 

This was a catastrophic event and all system durability was proven pretty well which i am happy about. There has never been a time in this current Datacenter where we have had complete power loss. We had a network hiccup a few years ago due to a misconfigured switch but that was a blip in comparison to today's real deal full dark outage. 

  • Like 1
Link to comment
Share on other sites

For non nerd version. I only put equipment I own on the red dots which are very specific facilities. Choosing to do it this way makes it cost effective and the most robust and reliable deployment possible. 

image.png

  • Hook 'Em 1
Link to comment
Share on other sites

Posted (edited)
34 minutes ago, immamac said:

This was a catastrophic event and all system durability was proven pretty well which i am happy about. There has never been a time in this current Datacenter where we have had complete power loss. We had a network hiccup a few years ago due to a misconfigured switch but that was a blip in comparison to today's real deal full dark outage. 


Don't worry about it. Shit happens. You don't owe us anything.

However, I'm about to get evil on those assholes in the 2025 recruiting thread who insist on discussing whether Katy is in Houston. Jeeze.

Edited by Hornius Emeritus
  • Hook 'Em 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.



×
×
  • Create New...