Monday, August 22, 2011

Fun with...

I don't even know how to google this one.

Today, I tried to log in to a client site. It's not a VPN, like the Cisco VPN; it's the RSA SecurID variety (if that makes a difference).

Basically, I log in with my network credentials and then enter a passcode which is generated from a pin I enter.

Friday was the last time I successfully logged into their system. Today, I was unable to get to the page.

tracert (Windows) and traceroute (linux) didn't do much for me.
  1     1 ms    <1 ms    <1 ms  Wireless_Broadband_Router.home [192.168.1.1]
  2     7 ms     6 ms     5 ms  L100.TAMPFL-VFTTP-73.verizon-gni.net [173.65.30.1]
  3    24 ms    27 ms    24 ms  G0-3-3-3.TAMPFL-LCR-22.verizon-gni.net [***.***.***.***]
  4     4 ms     7 ms     4 ms  so-2-0-0-0.TPA01-BB-RTR2.verizon-gni.net [130.81.28.214]
  5    11 ms     9 ms    10 ms  0.ge-3-2-0.XL4.MIA4.ALTER.NET [152.63.1.153]
  6    78 ms    70 ms    69 ms  0.ge-4-1-0.XT2.DEN4.ALTER.NET [152.63.114.201]
  7    70 ms    69 ms    71 ms  POS7-0-1.GW10.DEN4.ALTER.NET [152.63.89.213]
  8    68 ms    69 ms    66 ms  internap-gw.customer.alter.net [152.179.104.90]
  9    67 ms    67 ms    68 ms  mpr2.den.ve3-bbnet2.pnap.net [216.52.40.72]
 10    72 ms    89 ms    70 ms  allmar-4.mpr2.den.pnap.net [66.151.163.98]
 11     *        *        *     Request timed out.
 12     *        *        *     Request timed out.
 13     *        *        *     Request timed out.
 14     *        *        *     Request timed out.
 15     *        *        *     Request timed out.
 16     *        *        *     Request timed out.
 17     *        *        *     Request timed out.
 18     *        *        *     Request timed out.
 19     *        *        *     Request timed out.
 20     *        *        *     Request timed out.
 21     *        *        *     Request timed out.
 22     *        *        *     Request timed out.
 23     *        *        *     Request timed out.
 24     *        *        *     Request timed out.
 25     *        *        *     Request timed out.
 26     *        *        *     Request timed out.
 27     *        *        *     Request timed out.
 28     *        *        *     Request timed out.
 29     *        *        *     Request timed out.
 30     *        *        *     Request timed out.
I have no idea what that means, but I seriously doubt that if it was working correctly I would see the request time outs.

Interestingly, if I just use the host, say ssl.oraclenerd.com, I get this:
  1    <1 ms    <1 ms    <1 ms  Wireless_Broadband_Router.home [192.168.1.1]
  2     3 ms     4 ms     6 ms  L100.TAMPFL-VFTTP-73.verizon-gni.net [***.***.***.***]
  3     4 ms     4 ms     4 ms  G0-3-3-4.TAMPFL-LCR-22.verizon-gni.net [130.81.110.222]
  4     4 ms     4 ms     4 ms  so-2-0-0-0.TPA01-BB-RTR2.verizon-gni.net [130.81.28.214]
  5    11 ms     9 ms     9 ms  0.ge-3-2-0.XL4.MIA4.ALTER.NET [152.63.1.153]
  6     *        *        *     Request timed out.
  7    21 ms    22 ms    23 ms  GigabitEthernet7-0-0.GW11.ATL5.ALTER.NET [152.63.80.53]
  8    51 ms    62 ms    40 ms  oraclenerd.alter.net [157.130.87.138]
  9    25 ms    27 ms    26 ms  connect.oraclenerd.com [10.115.2.235]
Doesn't seem to be an issue there.

More information
I could not connect from any of my virtual machines, nor my host. Well, that's not quite true. While logged in through a Cisco VPN, I could successfully connect to this page (I never tried logging in, I was just testing to see if I could pull up the page). When I disconnected from the VPN, I could no longer access the page.

Over the weekend we had a power failure and I had to reboot the router. Also over the weekend I added an entry to my hosts file for a local connection.

If you know what this is, I will be indebted to you for life. I am sure I could fix it, if only I knew what was wrong. T-Shirts are on the table.

At this point, I'd even offer up my first born. ;)

Update 08/24/2011 12:19 AM EST
Finally resolved the issue. Funny, the last thing I did was call the ISP. I think I took the "blame the user" mentality a tad too far.

I have Verizon Fios, which rocks.



Support is pretty darn good too.

We got the service way back in 2007. This is only the second problem I have had. The last was just a couple of months ago (foreshadowing?).

They bounced the router remotely, then it just started to cycle on and off. They ordered me a new one. 24 to 48 hours. I couldn't wait. I went to the Verizon store down the street and picked up a new (old, used) one for a temporary fix. Tomorrow I'll be getting the new (new) one.

Anyway, plugged it in, navigated to the troublesome page, and voilĂ !

Much thanks to Martin who walked me through various steps (see comments) over IM yesterday. This isn't the first time he's helped me out either. Top notch guy.

5 comments:

jpiwowar said...

Lack of traceroute info for the other end doesn't necessarily signify much. They might be filtering those packets regardless or whether you've connected successfully.

Random flailing:
1) Any chance your client's network nerds have blacklisted a block of IPs that includes the one you're coming from?
1a) The fact that you can get to the page while connected via a "thick" VPN client (Cisco) means that the service is functional, at least. If this client supports other remote people (salespeeps, teleworkers), they might have a troubleshooting process that will allow them to look at logs on the VPN server and see why you're getting blocked.
2) Are you getting anything remotely (hah) useful in the form of a error code in the browser, or just a generic 404?
3) Did your weekend router reboot by any chance wipe out any settings related to VPN tunneling? Probably not, if Cisco VPN works, but maybe there are ports you needed to open that you opened once and forgot about?

oraclenerd said...

@jp

1. No packets are being dropped according to them.
1a. I'm on the horn with them now. They're a bit stumped. I have conceded that this is self-inflicted.
2. From Chrome I get "Error 118 (net::ERR_CONNECTION_TIMED_OUT): The operation timed out." Similar message from FF.
3. I've wondered that myself. Honestly, I don't know.

Martin Berger said...

I had some fun with Chet on this, so let's be more precise:
1) no IPs are blacklisted (on IP layer) as ping worked. - IP is fine (at east), but TCP to port 443 isn't.
1a) it's a https connection, which does not work (proved by wireshark -- no packages returned) - was it ever said Cisco VPN (IPSec?) to the same host works !?
2) no error code from the web-server, as no packages are returned (see 1a))
3) as the router has no static IP, this might has changed for sure. other changes are not clear right now.

To test the router, a 'tracing' on the public interface might be needed. regarding of it's setup it is somewhere between complex and impossible for the home-nerd.

Joel Garry said...

Did your customer originally send you a file with a token in it? You may need to reexecute that file to import the token.

As I understand it, RSA keys work by salting with the token, generating a new token every 30 seconds. So if you've lost the correct value of the token, it will never work. You may need to have your customer regenerate the file.

I'm guessing the blocking happens when the key tries to get through a firewall, which just acts like you never got anywhere. It ain't secure unless it hurts.

There may also be some setting on your browser that has gone to Richard Branson's island house. If they sent instructions that specified any settings, check them again.

oraclenerd said...

@joel

Interesting...I wonder if that would be the case? No error messages though; I guess from a security standpoint, the less information someone has the better. From a "debugging" standpoint, I think that would be evil. Interesting concept though.

In my case, it appears to be a dying router. Replacing it today did the track.