Pelican Parts
Parts Catalog Accessories Catalog How To Articles Tech Forums
Call Pelican Parts at 888-280-7799
Shopping Cart Cart | Project List | Order Status | Help



Go Back   Pelican Parts Forums > Miscellaneous and Off Topic Forums > Off Topic Discussions


Reply
 
LinkBack Thread Tools Rating: Thread Rating: 3 votes, 2.33 average.
Author
Thread Post New Thread    Reply
Registered
 
Rick Lee's Avatar
 
Join Date: Jul 2001
Location: Cave Creek, AZ USA
Posts: 44,469
Garage
How to get around a roboto.txt block?

I need to go onto www.mortgagequestions.com and cannot. I get a "Page cannot be displayed" message both on my work and personal laptops. When I do a Google search of the URL, it comes up with a robot.txt message as to why I can't get onto the site. I have no idea how to get around this and have never seen it before. Any ideers?

__________________
2022 BMW 530i
2021 MB GLA250
2020 BMW R1250GS
Old 12-15-2014, 05:58 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #1 (permalink)
You do not have permissi
 
john70t's Avatar
 
Join Date: Aug 2001
Location: midwest
Posts: 39,855
Javascripts might be blocked on the browser?
Not sure.

Checking to see if this indeed a legitimate website:
Whois.com - Domain Names & Identity for Everyone
=Bunch of hosting advertisements

www.allwhois.com
redirects to
https://www.markmonitor.com/
=blahblahblah"to protect the reputations and revenue of the world's leading brands"
Old 12-15-2014, 06:38 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #2 (permalink)
The Unsettler
 
stomachmonkey's Avatar
 
Join Date: Dec 2002
Location: Lantanna TX
Posts: 23,885
Send a message via AIM to stomachmonkey
Quote:
Originally Posted by Rick Lee View Post
I need to go onto www.mortgagequestions.com and cannot. I get a "Page cannot be displayed" message both on my work and personal laptops. When I do a Google search of the URL, it comes up with a robot.txt message as to why I can't get onto the site. I have no idea how to get around this and have never seen it before. Any ideers?
What browser?
__________________
"I want my two dollars"
"Goodbye and thanks for the fish"
"Proud Member and Supporter of the YWL"
"Brandon Won"
Old 12-15-2014, 06:54 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #3 (permalink)
Registered
 
Rick Lee's Avatar
 
Join Date: Jul 2001
Location: Cave Creek, AZ USA
Posts: 44,469
Garage
IE and Windows 7.
Old 12-15-2014, 06:55 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #4 (permalink)
Back in the saddle again
 
masraum's Avatar
 
Join Date: Oct 2001
Location: Central TX west of Houston
Posts: 55,927
Quote:
Originally Posted by john70t View Post
Javascripts might be blocked on the browser?
Not sure.
Nope, that's not it. When I try to access the site, I get a login page, and that's with Java blocked. When I enabled java, it did the same thing.

I assume you're going to need to login?

Can you access the page using a smart phone?

You say that you've tried both your PC and your work laptop. Both through the same Internet connection, probably at your house? Can you VPN into work and then try to access it. I'm thinking that it's either your ISP or your home router.
__________________
Steve
'08 Boxster RS60 Spyder #0099/1960
- never named a car before, but this is Charlotte.
'88 targa SOLD 2004 - gone but not forgotten
Old 12-15-2014, 07:02 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #5 (permalink)
Back in the saddle again
 
masraum's Avatar
 
Join Date: Oct 2001
Location: Central TX west of Houston
Posts: 55,927
If you need to log in to the website, I would absolutely NOT use this, but if you just want to do this for testing from your home network or PC to pull the page up without logging in, use this page

https://hide.me/en/proxy
or
https://www.filterbypass.me/
or
This one may or may not work for that website
Free web proxy – unblock any website – webproxy.net

Again, if you use one of the links above, and you login, I can't guaranty that those pages won't save your login info or be able to be hacked to give up your login info. But, I would say that they are fine/great for testing.
__________________
Steve
'08 Boxster RS60 Spyder #0099/1960
- never named a car before, but this is Charlotte.
'88 targa SOLD 2004 - gone but not forgotten
Old 12-15-2014, 07:09 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #6 (permalink)
 
Back in the saddle again
 
masraum's Avatar
 
Join Date: Oct 2001
Location: Central TX west of Houston
Posts: 55,927
Quote:
Originally Posted by Rick Lee View Post
I need to go onto www.mortgagequestions.com and cannot. I get a "Page cannot be displayed" message both on my work and personal laptops. When I do a Google search of the URL, it comes up with a robot.txt message as to why I can't get onto the site. I have no idea how to get around this and have never seen it before. Any ideers?
Can you paste the message that you get when you try to type the page into a browser and also the message that you get when you try to google search for it?
__________________
Steve
'08 Boxster RS60 Spyder #0099/1960
- never named a car before, but this is Charlotte.
'88 targa SOLD 2004 - gone but not forgotten
Old 12-15-2014, 07:10 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #7 (permalink)
Back in the saddle again
 
masraum's Avatar
 
Join Date: Oct 2001
Location: Central TX west of Houston
Posts: 55,927
Yeah, if I google search for www.mortgagequestions.com, it gives me lots of results. My guess is that something you've got is blocking access, ISP, home router, or possibly even AV (though that seems less likely unless your home and work PC use the same software.

If you try a smartphone, make sure your smart phone is on it's own data plan and not just surfing over your home wifi.
__________________
Steve
'08 Boxster RS60 Spyder #0099/1960
- never named a car before, but this is Charlotte.
'88 targa SOLD 2004 - gone but not forgotten
Old 12-15-2014, 07:12 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #8 (permalink)
Registered
 
Rick Lee's Avatar
 
Join Date: Jul 2001
Location: Cave Creek, AZ USA
Posts: 44,469
Garage
I can't use my iPhone for it, as I need to type stuff in their templates and upload documents. WTF?
__________________
2022 BMW 530i
2021 MB GLA250
2020 BMW R1250GS
Old 12-15-2014, 08:12 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #9 (permalink)
The Unsettler
 
stomachmonkey's Avatar
 
Join Date: Dec 2002
Location: Lantanna TX
Posts: 23,885
Send a message via AIM to stomachmonkey
Quote:
Originally Posted by Rick Lee View Post
IE and Windows 7.
Well it's one of those stupid sites that is "optimized" for certain browsers, translation, we don't know how to make it work right for all browsers" or " we used Microsoft Frontpage and are too ****ing lazy to bother accommodating anyone not using XXXXX"

It wants IE 9 or better or Firefox.

Their robots.txt does not appear to be the real issue.

Only contents are

User-agent: *
Disallow: /info
Disallow: /home

What that says is for all bots do not allow them to read /home and /info pages.

Home is normally the primary landing page, not sure why someone would have it under a specific directory, industry standard is xx.com/index.html where index.html is the default page a browser will read upon hitting xx.com

Try hitting https://www.mortgagequestions.com/sso/mq/login.jsp
__________________
"I want my two dollars"
"Goodbye and thanks for the fish"
"Proud Member and Supporter of the YWL"
"Brandon Won"
Old 12-15-2014, 08:24 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #10 (permalink)
Registered
 
Rick Lee's Avatar
 
Join Date: Jul 2001
Location: Cave Creek, AZ USA
Posts: 44,469
Garage
That link doesn't work (for me) either. I'll switch over to Firefox tomorrow and report back. Thanks for the suggestions.
__________________
2022 BMW 530i
2021 MB GLA250
2020 BMW R1250GS
Old 12-15-2014, 08:52 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #11 (permalink)
Registered User
 
Join Date: Sep 2012
Location: Växjö Sweden/Hannover Germany
Posts: 1,135
robots.txt has nothing to do with that issue.
robots.txt is only a RECOMMENDATION for the crawlers/spiders (they do not HAVE to follow that, but good ones do), it does not affect a browser at all.

On FF and Chrome it works for me. I get a HUGE link in the browsers URL bar, like this:
Code:
https://www.mortgagequestions.com/sso/mq/login.jsp?TYPE=33554433&REALMOID=06-69d40195-9749-498e-841a-977f492bac26&GUID=&SMAUTHREASON=0&METHOD=GET&SMAGENTNAME=mortquest-secure@lb1b2cwb10p&TARGET=$SM$HTTPS%3a%2f%2fwww.mortgagequestions.com%2fsso%2fmq2%2flogin.jsp%3fTYPE%3d33554433%26REALMOID%3d06-bdc806c4-8355-4c2f-b4e0-ac3ecc5c7494%26GUID%3d%26SMAUTHREASON%3d0%26METHOD%3dGET%26SMAGENTNAME%3dmortquest-secure%40lb1b2cwb10p%26TARGET%3d$$SM$$HTTPS$%3a$%2f$%2fwww$.mortgagequestions$.com$%2fhome$%2flandscape$%3fjpid$%3dHome$%26cid$%3d15882
Old 12-15-2014, 10:41 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #12 (permalink)
Registered
 
motion's Avatar
 
Join Date: Apr 2000
Location: Mid-life crisis, could be anywhere
Posts: 10,382
PHH Mortgage.... GRRRRRRRRRR
__________________
'95 993 C4 Cabriolet
Bunch of motorcycles
Old 12-16-2014, 03:10 AM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #13 (permalink)
Get off my lawn!
 
GH85Carrera's Avatar
 
Join Date: Nov 2007
Location: Oklahoma
Posts: 84,848
Garage
Just as information I tried the link. It worked fine and I saw where to log in and a pic of a little girl at play.

I am using Firefox and WinXP. I love Firefox. I was slow to get away from IE and I am sorry I waited so long. Firefox is way faster and work.
__________________
Glen
49 Year member of the Porsche Club of America
1985 911 Carrera; 2017 Macan
1986 El Camino with Fuel Injected 350 Crate Engine
My Motto: I will never be too old to have a happy childhood!
Old 12-16-2014, 04:16 AM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #14 (permalink)
Registered
 
Rick Lee's Avatar
 
Join Date: Jul 2001
Location: Cave Creek, AZ USA
Posts: 44,469
Garage
I was able to get past the login page yesterday on both computers. But after that I got the "page cannot be displayed" message. I can't put Firefox on my work computer, but will do it on home laptop tonight.
__________________
2022 BMW 530i
2021 MB GLA250
2020 BMW R1250GS
Old 12-16-2014, 05:15 AM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #15 (permalink)
Moderator
 
Z-man's Avatar
 
Join Date: Feb 2001
Location: NJ, USA
Posts: 9,628
Garage
Assuming the website answers questions about mortgages, with all the trouble you are going through to get to the website -- why not just call up a local real estate broker and a mortgage person?

Just sayin,
-Z
__________________
2010 Cayman S - 12-2020 -
2014 MINI Cooper S Coupe - 05-17 - 05-21
1989 944S2 - 06-01 - 01-14
Carpe Viam.
<><
Old 12-16-2014, 05:59 AM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #16 (permalink)
Registered
 
Rick Lee's Avatar
 
Join Date: Jul 2001
Location: Cave Creek, AZ USA
Posts: 44,469
Garage
We're a little beyond that, as I've had my mortgage with HSBC for 10 yrs. and PHH services it. I'm selling the house and need to upload some docs there. Calling their 800# just gets me some kid in India who tells me to go to that website for everything, which is slightly less frustrating than having a conversation with one of those guys.
__________________
2022 BMW 530i
2021 MB GLA250
2020 BMW R1250GS
Old 12-16-2014, 06:06 AM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #17 (permalink)
Moderator
 
Z-man's Avatar
 
Join Date: Feb 2001
Location: NJ, USA
Posts: 9,628
Garage
Quote:
Originally Posted by Rick Lee View Post
We're a little beyond that, as I've had my mortgage with HSBC for 10 yrs. and PHH services it. I'm selling the house and need to upload some docs there. Calling their 800# just gets me some kid in India who tells me to go to that website for everything, which is slightly less frustrating than having a conversation with one of those guys.
Gotcha. And I laughed... It could be worse -- it could be a Motor Vehicle Services Representative on the phone....


-Z
__________________
2010 Cayman S - 12-2020 -
2014 MINI Cooper S Coupe - 05-17 - 05-21
1989 944S2 - 06-01 - 01-14
Carpe Viam.
<><
Old 12-16-2014, 06:34 AM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #18 (permalink)
Registered
 
Rick Lee's Avatar
 
Join Date: Jul 2001
Location: Cave Creek, AZ USA
Posts: 44,469
Garage
Just downloaded Firefox, got past the logon page for mortgagequestions.com, got to the "register this computer" page and then it went to "unable to connect." WTF? I really need to get on this and nothing is working. Should I try Google Chrome next?
__________________
2022 BMW 530i
2021 MB GLA250
2020 BMW R1250GS
Old 12-16-2014, 04:23 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #19 (permalink)
Back in the saddle again
 
masraum's Avatar
 
Join Date: Oct 2001
Location: Central TX west of Houston
Posts: 55,927
OK, so you get the home page up and then login, and then the failure occurs? That's different than not getting there at all.

Since you are having issues in the new browser as well as the old, try clearing your java cache.
Quote:
Delete Temporary Files through the Java Control Panel

In the Java Control Panel, under the General tab, click Settings under the Temporary Internet Files section. ...
Click Delete Files on the Temporary Files Settings dialog. ...
Click OK on the Delete Files and Applications dialog.
The whole point of sending you to the page via your cell phone wasn't for you to do all of your work via your cell phone, but to see if bypassing your ISP and Internet router "fixed" the problem. If you can login with your phone via your data plan, then the problem may be your ISP or PC or Internet router. If you can't get there via your phone while on it's data plan, then maybe there's a problem with the website.

If using your phone works, then link your phone to your wifi and try again. If it still works, then you know the problem isn't your ISP or your Internet router. At that point you would be certain that the problem was with your laptop (both of them apparently).

If we are certain that it's the laptops, then the easiest and first thing to do would be to clear the browser cache (unlikely since you just installed a new browser) or your java cache (more likely since the new browser could be using the same old java cache as the other browser. Maybe the site changed something in their java and your old cached java is causing the problem.

__________________
Steve
'08 Boxster RS60 Spyder #0099/1960
- never named a car before, but this is Charlotte.
'88 targa SOLD 2004 - gone but not forgotten
Old 12-16-2014, 04:37 PM
  Pelican Parts Catalog | Tech Articles | Promos & Specials    Reply With Quote #20 (permalink)
Reply


 


All times are GMT -8. The time now is 06:09 AM.


 
Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
Search Engine Optimization by vBSEO 3.6.0
Copyright 2025 Pelican Parts, LLC - Posts may be archived for display on the Pelican Parts Website -    DMCA Registered Agent Contact Page
 

DTO Garage Plus vBulletin Plugins by Drive Thru Online, Inc.