Ask HN: Is Google Compute down?

by hellcow on 11/12/2018, 9:47 PM with 74 comments

I'm not able to ssh into any of my boxes or access any of the sites, yet my status monitor isn't showing downtime. Spotify is also down for me, which is another GCP customer.

I'm in Los Angeles but the servers are hosted on us-central1

  • by sethvargo on 11/12/2018, 10:23 PM

    Hi all - Seth from Google here. Our team is aware and we are working on mitigation. In short, a third party telco provider is advertising on one of our IP blocks. Unfortunately that's all the information I can share at this time.

  • by regnerba on 11/12/2018, 9:54 PM

    Google IPs seem to be being routed to China for us.

    We have servers in San Jose that cannot access Google services. Trace route shows everything going to China when leaving the San Jose data center. We can access the same services from Vancouver just fine.

  • by jamalex on 11/12/2018, 10:23 PM

    Despite the subdomain, the IP for ChinaTelecom-gw.transtelecom.net (217.150.59.249) seems to be based in Russia, as does the carrier: https://en.wikipedia.org/wiki/TransTelekom

  • by xolox on 11/12/2018, 10:35 PM

    Reading through the comments here I'm recognizing "China Telecom" from an article on a BGP hijack that was published about a week ago, I still had the article open in my browser:

    https://arstechnica.com/information-technology/2018/11/stran...

    In another comment in this thread I read:

    > Seems like its time to start or accelerate a working group on secure BGP.

    Indeed things can't go on like this for much longer...

  • by davismwfl on 11/12/2018, 10:04 PM

    I am on the East Coast, in Florida and seeing the same thing with traffic heading to China, lots of "chinatelecom-gw.transtelecom.net" in traceroutes I have never seen prior.

  • by CydeWeys on 11/12/2018, 10:32 PM

    We urgently need a solution for routing traffic to IP addresses that is better than BGP.

  • by aviv on 11/12/2018, 10:29 PM

    Funny, a day after I posted this...

    https://news.ycombinator.com/item?id=18429099

    Is our first time actually rolling over the entire stack to AWS - and it worked!

    GCP outage currently is massive, can't even use other regions.

    Edit: This also affected AWS Oregon region earlier. I do not know how yet, but they too were unreachable briefly. Seems to be okay now.

  • by infogulch on 11/12/2018, 10:35 PM

    So... what's the current state of a secure BGP? I feel like this in the top 3 security threats to the whole of the internet.

  • by on 11/12/2018, 10:12 PM

    undefined

  • by TodayIsTheDay on 11/12/2018, 9:53 PM

    Does anybody else have chinatelecom-gw.transtelecom.net [217.150.59.249] in the traceroute for www.google.com

  • by dickfickling on 11/12/2018, 9:50 PM

    yeah, GCP is having a serious outage. Our site is down, so's Pivotal Tracker

    Edit: We're also in Los Angeles, connecting to us-central1. Seems to be a pattern?

  • by syogi on 11/12/2018, 9:49 PM

    I'm in Los Angeles and I can access my GCP Console but I can't access Google services like google.com or Maps or Gmail.

    EDIT: Some services are intermittently responsive. I had ~5 minutes of no access to anything. Some are slowly coming back.

  • by fxdoublecute on 11/12/2018, 10:09 PM

    we manage services deployed in every GCE region, and our monitoring in London is reporting every GCE region having intermittent connectivity. no problems with our services in the other major clouds (we use basically all of them)

  • by vamos_davai on 11/12/2018, 10:16 PM

    I have trouble accessing YouTube. I live in Sherman Oaks (a town of Los Angeles).

  • by gsibble on 11/12/2018, 9:57 PM

    Can confirm many IPs are being sent to China.

  • by cobookman on 11/12/2018, 10:25 PM

    There's a current BGP prefix hijacking issue currently being mitigated.

  • by bifrost on 11/12/2018, 11:13 PM

    I hate to break it to everyone, but the technology to filter this sorta thing has existed for a very long time, but people often don't use it. Most of the time this sort of thing is accidental (IE: operator error)so a lot of operators kinda ignore it. Check out "IRR Power Tools" if you're interested.

  • by jamalex on 11/12/2018, 10:11 PM

    Same thing here in San Diego. Traceroute to spotify.com going through LA, San Jose, NY, London, Amsterdam, Frankfurk, "mskn17ra-lo1.transtelecom.net", then ChinaTelecom-gw.transtelecom.net.

  • by RayHawk on 11/12/2018, 10:28 PM

    I'm getting the same thing. Servers are in us-east1 and tracert is ending at chinatelecom-gw.transtelecom.net [217.150.59.249]

  • by xstephen95x on 11/12/2018, 11:03 PM
  • by johnnyballgame on 11/12/2018, 9:50 PM

    YouTube and Spotify unresponsive here.

  • by linuxbuzz on 11/12/2018, 11:06 PM

    World War III on the Internet front!

  • by jetforme on 11/12/2018, 10:29 PM

    I'm in LA and seeing similar routing through chinatelecom-gw.transtelecom.net

  • by kenhwang on 11/12/2018, 10:02 PM

    Might be related to: https://status.cloud.google.com/incident/cloud-networking/18...

    Also in LA, had intermittent issues with google.com and Spotify all morning.

    edit: linked to wrong issue

  • by dejaime on 11/12/2018, 10:36 PM

    Well, my instance seems to be working

  • by ntq on 11/12/2018, 9:49 PM

    We are able to connect to some of our services via LTE but not a local wifi network

  • by on 11/12/2018, 10:08 PM

    undefined