Yet another reason to hate Google

[RJM62]

So I was doing some routine checking to see one of my clients' sites was coming up on Google Search. Things looked good: My client was number one.

Until I noticed that Google search, for reasons unknown, had inserted a trailing slash behind the page URL!!! See the screenshot if you don't believe me.

Now what that does to the site is loads the page, but without the stylesheet, because the link is no longer correct. So the page comes up white and inline, with no style elements, like the second screenshot.

Now, if it were just the style problem, it would be easy enough to change the link to the stylesheet to an absolute (rather than a relative) link. But what about all the other stuff that's linked from the pages? None of that would work, either, because all the internal links pointed to imaginary directories.

I quickly did a few more searches, and found that at least one other page had been indexed the same way -- with a trailing slash.

There is nothing on the site (nor its mobile version) that has trailing slashes after the filename that Google could have picked up and indexed, nor do directories with those names exist. Nor does any other search engine that I tried produce this error. Nor do I have any idea whatsoever why it happened.

All I know is that I'm royally pi$$ed.

In the meantime, for at least a day (according to Google's cache), visitors have been seeing those ugly, css-less pages.

Anyway, I kludged out a fix by putting the following in .htaccess:

# Redirect to remove trailing slashes from freaking google search results
# If URL ending with a slash does not resolve to an existing directory
RewriteCond %{REQUEST_FILENAME} !-d
# Externally redirect to remove trailing slash
RewriteRule ^(.+)/$ http://www.[domain].com/$1 [R=301,NC]

Which seems to work, although I imagine it'll increase overhead a bit.

What the heck is up with Google lately? From the pi$$-poor CTR and revenue performance of Adsense since they started using user data for ad selection, to their incessant and unrelenting attempts to get my cell phone number by any means necessary (Did you know you can't even view Google+ pages on a mobile device without logging in to your Google account?), to things like this.

They used to be a company I admired. Now I seriously wish they would just go belly-up.

/rant

Okay, thanks. I feel better now.

-Rich

 

[zaitcev]

Just return 404 for the trailing slash. Only make sure the root URL redirects where it should.

 

[jsstevens]

Maybe uselessly paranoid, but did you notice the paid ad link to the same site just above was correct?


Sent from my iPad using Tapatalk HD

 

[TangoWhiskey]

Do you have any other 301 redirects in your rules for this site?

 

[ChrisK]

Maybe uselessly paranoid, but did you notice the paid ad link to the same site just above was correct?

That's because sponsored links are entered by the advertiser manually. Search results are automatically discovered and indexed by Google.

It seems to me that "someone" has an incorrect link to your site somewhere and Google scooped it up. Have you submitted a sitemap for your site using webmaster tools?

Incidentally I think using mod_rewrite was a good idea.

 

[RJM62]

Just return 404 for the trailing slash. Only make sure the root URL redirects where it should.

The 301 should eventually update Google's database (permanent redirect), as well as getting the visitors to the right pages.

-Rich

 

[RJM62]

Maybe uselessly paranoid, but did you notice the paid ad link to the same site just above was correct?

Those are manually entered by the client when he sets up the ads. Google just copies them verbatim.

-Rich

 

[RJM62]

Do you have any other 301 redirects in your rules for this site?

Yes, but they're ancient from before the site was rewritten in PHP, and they're all correct.

-Rich

 

[RJM62]

That's because sponsored links are entered by the advertiser manually. Search results are automatically discovered and indexed by Google.

It seems to me that "someone" has an incorrect link to your site somewhere and Google scooped it up. Have you submitted a sitemap for your site using webmaster tools?

Incidentally I think using mod_rewrite was a good idea.

Thanks. I do have a sitemap, and the links are correct on it. I just noticed that I haven't added a couple of new pages, however, so I have to update it.

I was wondering whether a competitor posted the bad link somewhere. But as far as I can tell, only two pages were affected. I would think a malicious individual would go further.

In any event, the rule should stop any further attempts of that sort, if it's the case.

-Rich

EDIT: Actually, the sitemap is up to date. For some reason, Firefox's "Find" function didn't work the first time (I wanted to read the sitemap.xml file online to see if it got corrupted). It worked when I refreshed the page, though.

 

[RJM62]

According to the raw logs, the only hits on the pages with the trailing slashes came through Google Search, and only about a third of the hits coming through Google Search had the slash. I guess it depends on which of Google's servers they came through.

-Rich

 

[Jack Spectre]

They used to be a company I admired.

Ha, when was that? Maybe the first few years of their operation, but they've turned into a powerful juggernaut with tentacles in EVERYTHING. They know that information is power, and they are acquiring it as fast as they can.

I don't use Google's products (except maybe Google Earth, which doesn't require an account or ads to run) and I get along just fine.

Humans don't handle absolute power very well.