I've been building a community-driven hunting and fishing site in Texas for the past year and a half. Since I have converted it to Wicket from ColdFusion, our search engine rankings have gone WAY UP. That's right, we're on the first page for tons of searches. Search for "texas hunting" - we're second under only the Texas Parks and Wildlife Association.
How? With Wicket? Yes - it requires a little more work. What I do is that for any link that I want Google to be able to follow, I have a subclass of Link specific to that. For instance, ViewThreadLink, which takes the ID for the link and a model (detachable) of the thread. Then I mount an IRequestTargetUrlCodingStrategy for each big category of things in my webapp. I've made several strategies that I use over and over, just giving them a different mount path and a different parameter to tell it what kind of article, etc, that it will match to. This is made easier because over 75% of the objects in our site are all similar enough that the extend from a base class that provides the basic functionality for an article / thread / etc that has a title, text, pictures, comments, the standard stuff. So, yes, it takes work. But that's okay - SEO always takes work. I also have given a lot of care to use good page titles, good semantic HTML and stuff things into the URL that don't have anything to do with locating the resource, but give the search engines a clue as to what the content is. Yes, some pages end up with a jsessionid - and I don't like it (example: http://www.google.com/search?hl=en&client=firefox-a&rls=com.ubuntu%3Aen-US%3Aofficial&q=%22south+texas+management+buck%22&btnG=Search). But, most don't because almost all of my links are bookmarkable. When the user clicks something that they can only do as a signed-in user, then it redirects them to the sign in page, they sign in, and are taken back to the page they were on. Then they can pick up, and I don't worry about bookmarkable URLs for anything that requires user-authentication (wizards to post a new listing, story, admin links, etc). Jeremy Thomerson TexasHuntFish.com On Thu, Apr 3, 2008 at 8:09 PM, Dan Kaplan <[EMAIL PROTECTED]> wrote: > Ok, at least I'm not missing anything. I understand the benefits it's > providing with its stateful framework. Developing a site with Wicket is > easier than with any other framework I've used. But this statefulness, > which makes websites so easy to develop, seems to be counter productive to > SEO: > > GoogleBot will follow and index stateful links. Worst case scenario, > these > actually become visible to google users and when they click the link it > takes them to an "invalid session" page. They think, "This site is > broken" > and move on to the next link of their search result. > > Another approach to solving this is to block all the stateful pages in my > robots.txt file. But how can I block these links in robots.txt since they > change per session? Is there any way to know what the url will resolve to > when googlebot tries to visit my site so I can tell it to disallow: > /?wicket:interface=:10:1::: and ?wicket:interface=:0:1::: and ...? > > > > -----Original Message----- > > From: Igor Vaynberg [mailto:[EMAIL PROTECTED] > > Sent: Thursday, April 03, 2008 5:45 PM > > To: users@wicket.apache.org > > Subject: Re: Removing the jsessionid for SEO > > > > On Thu, Apr 3, 2008 at 5:31 PM, Dan Kaplan <[EMAIL PROTECTED]> > > wrote: > > > Ok I did a little preliminary research on this. Right now > > PagingNavigator > > > uses PagingNavigationLink's to represent its page. This extends > Link. > > I'm > > > supposed to override PagingNavigator's newPagingNavigationLink() > method > > to > > > accomplish this (I think) but past that, this isn't very > > straightforward to > > > me. > > > > > > Do I need to create my own BookmarkablePagingNavigationLink? When I > > do... > > > what next? I really don't know enough about bookmarkablePageLinks to > > do > > > this. Right now, all the magic happens inside PagingNavigationLink. > > Won't > > > I have to move all that logic into the WebPage that I'm passing into > > > BookmarkablePagingNavigationLink? This seems like a lot of work. Am > I > > > missing something critical? > > > > no, you are not missing anything. you see, when you go stateless, like > > what you want, then you have to recreate all the magic stuff that > > makes stateful links Just Work. Without state you are back to the > > servlet/mvc programming model: you have to encode the state that you > > want into the link, then on the trip back decode it, recreate > > something from it, and then apply that something onto the components. > > This is the crapwork that wicket does for you usually. > > > > -igor > > > > > > > > > > > > > > -----Original Message----- > > > > From: Igor Vaynberg [mailto:[EMAIL PROTECTED] > > > > > > > > > > Sent: Thursday, April 03, 2008 3:40 PM > > > > To: users@wicket.apache.org > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > you subclass the pagenavigator and make it use bookmarkable links > > > > also. it has factory methods for all the links it uses. > > > > > > > > -igor > > > > > > > > > > > > On Thu, Apr 3, 2008 at 3:36 PM, Dan Kaplan <[EMAIL PROTECTED] > > > > > > wrote: > > > > > I wasn't talking about the links that are on the list (I already > > make > > > > those > > > > > bookmarkable). I'm talking about the links that the Navigator > > > > generates. > > > > > How do I make it so page 2 is bookmarkable? > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Igor Vaynberg [mailto:[EMAIL PROTECTED] > > > > > > > > > > > > > > > Sent: Thursday, April 03, 2008 3:30 PM > > > > > To: users@wicket.apache.org > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > instead of > > > > > > > > > > item.add(new link("foo") { onclick() }); > > > > > > > > > > do > > > > > > > > > > item.add(new bookmarkablepagelink("foo", page.class)); > > > > > > > > > > -igor > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 3:28 PM, Dan Kaplan > > <[EMAIL PROTECTED]> > > > > wrote: > > > > > > How? I asked how to do it before and nobody suggested this as > a > > > > > > possibility. > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Igor Vaynberg [mailto:[EMAIL PROTECTED] > > > > > > Sent: Thursday, April 03, 2008 3:26 PM > > > > > > To: users@wicket.apache.org > > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > > > dataview can work in a stateless mode, just use bookmarkable > > links > > > > inside > > > > > it > > > > > > > > > > > > -igor > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 3:22 PM, Dan Kaplan > > <[EMAIL PROTECTED]> > > > > > wrote: > > > > > > > Regardless, at the very least this makes your site look > > "weird" > > > > and > > > > > > > unprofessional when google puts a jsessionid on your url. > > There > > > > has > > > > > got > > > > > > to > > > > > > > be some negative effect when google visits it the second > > time and > > > > the > > > > > > > jsessionid has changed but it sees the same exact content. > > Worst > > > > > case, > > > > > > > it'll think you're trying to trick it. > > > > > > > > > > > > > > About those 404s, I'm finding that with the fix I provided > I > > > > don't get > > > > > a > > > > > > > 404, but the links refresh the page I'm already on. IE: > If > > I'm > > > > on A, > > > > > and > > > > > > a > > > > > > > link to B is non-bookmarkable, clicking B refreshes A. > > > > > > > > > > > > > > This issue is very disconcerting to me. It's one of the > > reasons > > > > I > > > > > wish > > > > > > that > > > > > > > DataView had an option to work in stateless mode. Cause > if > > I ban > > > > > cookies > > > > > > > and Googlebot visits my home page (with a navigator on > it), > > it'll > > > > try > > > > > to > > > > > > > follow all these page links and from its perspective, they > > all > > > > lead > > > > > back > > > > > > to > > > > > > > the first page. So it's kinda a catch-22: Include the > > jsessionid > > > > in > > > > > the > > > > > > > urls and get bad SEO or remove the jsessionid and get bad > > SEO :( > > > > > > > > > > > > > > Perhaps the answer to my prayers is a combination of the > > > > > noindex/nofollow > > > > > > > meta tag with a sitemap.xml. I'm thinking I can put a > > nofollow > > > > on the > > > > > > home > > > > > > > page (so googlebot doesn't try to follow the navigator > > links) and > > > > use > > > > > the > > > > > > > sitemap.xml to point out the individual pages I want it to > > index. > > > > > > > > > > > > > > > > > > > > > Matej: can you go into more detail about your hybrid URL > > > > statement? > > > > > > Won't > > > > > > > google index, for example, /home and /home.1 if I use it? > > When > > > > it > > > > > > follows > > > > > > > the next page, won't the url become /home.1.2 or > something? > > That > > > > .2 > > > > > is a > > > > > > > page version: If google indexes that and tries to visit it > > again, > > > > > won't > > > > > > it > > > > > > > report about an invalid session? > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Matej Knopp [mailto:[EMAIL PROTECTED] > > > > > > > Sent: Thursday, April 03, 2008 11:10 AM > > > > > > > To: users@wicket.apache.org > > > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > > > > > On the other hand, crawling non-bookmarkable pages is not > > very > > > > useful > > > > > > > anyway, since ?wicket:interface url will always get page > > expired > > > > when > > > > > > > you click on the result. > > > > > > > > > > > > > > However, preserving session makes lot of sense with hybrid > > url. > > > > Google > > > > > > > remembers the original url (without page instance) while > > indexing > > > > the > > > > > > > real page (after redirect). > > > > > > > > > > > > > > I think though that the crawler is quite advanced. I'm > would > > > > think it > > > > > > > supports cookies (at least JSESSIONID) as well as it > > evaluates > > > > some of > > > > > > > the javascript on page. > > > > > > > > > > > > > > -Matej > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 6:56 PM, Igor Vaynberg > > > > > <[EMAIL PROTECTED]> > > > > > > > wrote: > > > > > > > > right. if you strip sessionid then all your > > nonbookmarkable > > > > urls > > > > > will > > > > > > > > resolve to a 404. that will probably drop your rank a > lot > > > > > faster.... > > > > > > > > > > > > > > > > -igor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 9:16 AM, Johan Compagner > > > > > <[EMAIL PROTECTED]> > > > > > > > wrote: > > > > > > > > > the problem is that then you have to have all > stateless > > > > pages. > > > > > Else > > > > > > > google > > > > > > > > > can't crawl your website. > > > > > > > > > And if that is the case then you could be completely > > > > stateless > > > > > so > > > > > > you > > > > > > > dont > > > > > > > > > have a session (id) to worry about at all. > > > > > > > > > > > > > > > > > > johan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 4:54 PM, Zappaterrini, Larry > < > > > > > > > > > [EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > > > > > > When Google asks to not have special treatment for > > their > > > > bot, > > > > > > they > > > > > > > are > > > > > > > > > > referring to content more than anything. Regarding > > the > > > > session > > > > > id > > > > > > > being > > > > > > > > > > coded in the URL, see the Technical guidelines > > section of > > > > > > Google's > > > > > > > > > > Webmaster Guidelines - > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.google.com/support/webmasters/bin/answer.py?answer=35769#desi > > > > > > > > > > gn > > > > > > > > > > > > > > > > > > > > It specifically recommends "allow(ing) search bots > > to > > > > crawl > > > > > your > > > > > > > sites > > > > > > > > > > without session IDs or arguments that track their > > path > > > > through > > > > > > the > > > > > > > > > > site." > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: Johan Compagner [mailto:[EMAIL PROTECTED] > ] > > > > > > > > > > Sent: Thursday, April 03, 2008 7:35 AM > > > > > > > > > > To: users@wicket.apache.org > > > > > > > > > > Subject: Re: Removing the jsessionid for SEO > > > > > > > > > > > > > > > > > > > > isnt google always saying that you shouldn't alter > > > > behavior of > > > > > > your > > > > > > > site > > > > > > > > > > depending of it is there bot or not? > > > > > > > > > > > > > > > > > > > > On Thu, Apr 3, 2008 at 1:00 PM, Artur W. > > > > <[EMAIL PROTECTED]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > igor.vaynberg wrote: > > > > > > > > > > > > > > > > > > > > > > > > also by doing what you have done users with > > cookies > > > > > disabled > > > > > > > wont be > > > > > > > > > > > > able to use your site... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In my opinion session id is a problem. Google > > index the > > > > same > > > > > > page > > > > > > > > > > again > > > > > > > > > > > and > > > > > > > > > > > again. > > > > > > > > > > > > > > > > > > > > > > About the users without cookies we can do like > > this: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > static class Unbuffered extends > WebResponse > > { > > > > > > > > > > > > > > > > > > > > > > private static final String[] > > botAgents > > > > = { > > > > > > > > > > "onetszukaj", > > > > > > > > > > > "googlebot", > > > > > > > > > > > "appie", "architext", > > > > > > > > > > > "jeeves", "bjaaland", > > "ferret", > > > > > > "gulliver", > > > > > > > > > > > "harvest", "htdig", > > > > > > > > > > > "linkwalker", "lycos_", > > "moget", > > > > > > > > > > "muscatferret", > > > > > > > > > > > "myweb", "nomad", > > > > > > > > > > > "scooter", > > > > > > > > > > > "yahoo!\\sslurp\\schina", > > > > "slurp", > > > > > > > "weblayers", > > > > > > > > > > > "antibot", "bruinbot", > > > > > > > > > > > "digout4u", > > > > > > > > > > > "echo!", "ia_archiver", > > > > "jennybot", > > > > > > > "mercator", > > > > > > > > > > > "netcraft", "msnbot", > > > > > > > > > > > "petersnews", > > > > > > > > > > > "unlost_web_crawler", > > "voila", > > > > > > "webbase", > > > > > > > > > > > "webcollage", "cfetch", > > > > > > > > > > > "zyborg", > > > > > > > > > > > "wisenutbot", "robot", > > "crawl", > > > > > "spider" > > > > > > }; > > > > > > > /* > > > > > > > > > > and > > > > > > > > > > > so on... */ > > > > > > > > > > > > > > > > > > > > > > public Unbuffered(final > > > > HttpServletResponse > > > > > res) > > > > > > { > > > > > > > > > > > super(res); > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > @Override > > > > > > > > > > > public CharSequence encodeURL(final > > CharSequence > > > > url) > > > > > { > > > > > > > > > > > return isAgent() ? url : > > > > super.encodeURL(url); > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > private static boolean isAgent() > { > > > > > > > > > > > > > > > > > > > > > > String agent = > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ((WebRequest)RequestCycle.get().getRequest()).getHttpServletRequest().ge > > > > > > > > > > tHeader("User-Agent"); > > > > > > > > > > > > > > > > > > > > > > for(String bot : > botAgents) > > { > > > > > > > > > > > if > > > > > > > (agent.toLowerCase().indexOf(bot) != > > > > > > > > > > -1) > > > > > > > > > > > { > > > > > > > > > > > return > > true; > > > > > > > > > > > } > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > return false; > > > > > > > > > > > } > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I didn't test this code but I do similar thing > in > > my > > > > old > > > > > > > application > > > > > > > > > > in > > > > > > > > > > > Spring and it works. > > > > > > > > > > > > > > > > > > > > > > Take care, > > > > > > > > > > > Artur > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > View this message in context: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.nabble.com/Removing-the-jsessionid-for-SEO- > > > > tp16464534p1646739 > > > > > > > > > > > > > > > > > > > > > > > > > > > > 6.html<http://www.nabble.com/Removing-the-jsessionid-for-SEO- > > > > tp16464534p1646 > > > > > > > 7396.html> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sent from the Wicket - User mailing list archive > at > > > > > Nabble.com. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > ----- > > > > ---- > > > > > > > > > > > To unsubscribe, e-mail: users- > > > > [EMAIL PROTECTED] > > > > > > > > > > > For additional commands, e-mail: > > > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ______________ > > > > > > > > > > > > > > > > > > > > The information contained in this message is > > proprietary > > > > > and/or > > > > > > > > > > confidential. If you are not the > > > > > > > > > > intended recipient, please: (i) delete the message > > and > > > > all > > > > > > copies; > > > > > > > (ii) do > > > > > > > > > > not disclose, > > > > > > > > > > distribute or use the message in any manner; and > > (iii) > > > > notify > > > > > the > > > > > > > sender > > > > > > > > > > immediately. In addition, > > > > > > > > > > please be aware that any message addressed to our > > domain > > > > is > > > > > > subject > > > > > > > to > > > > > > > > > > archiving and review by > > > > > > > > > > persons other than the intended recipient. Thank > > you. > > > > > > > > > > _____________ > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > ----- > > > > ---- > > > > > > > > > > To unsubscribe, e-mail: users- > > > > [EMAIL PROTECTED] > > > > > > > > > > For additional commands, e-mail: users- > > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > --- > > > > > > > > To unsubscribe, e-mail: users- > > [EMAIL PROTECTED] > > > > > > > > For additional commands, e-mail: users- > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Resizable and reorderable grid components. > > > > > > > http://www.inmethod.com > > > > > > > > > > > > > > > ------------------------------------------------------------ > > ----- > > > > ---- > > > > > > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > > > > > > For additional commands, e-mail: users- > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > ----- > > > > ---- > > > > > > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > > > > > > For additional commands, e-mail: users- > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------- > > ----- > > > > - > > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------- > > ----- > > > > - > > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > --- > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > --- > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
