Hi Karl, I tried modifying the test code as you suggested.
When I tried with empty credentials NtlmPasswordAuthentication pa = new NtlmPasswordAuthentication("","",""); SmbFile smbconnection = new SmbFile("smb://" + "lon-data-arx.global.arup.com" + "/" + "stg-jobs$",pa); I got the same exception as before *Exception in thread "main" jcifs.smb.SmbAuthException: Logon failure: unknown us er name or bad password. at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:546) at jcifs.smb.SmbTransport.send(SmbTransport.java:663) at jcifs.smb.SmbSession.sessionSetup(SmbSession.java:390) at jcifs.smb.SmbSession.send(SmbSession.java:218) at jcifs.smb.SmbTree.treeConnect(SmbTree.java:176) at jcifs.smb.SmbFile.doConnect(SmbFile.java:911)* ...... When I tried with NtlmPasswordAuthentication.ANONYMOUS SmbFile smbconnection = new SmbFile("smb://" + "lon-data-arx.global.arup.com" + "/" + "stg-jobs$",NtlmPasswordAuthentication.ANONYMOUS); I got a different exception *Exception in thread "main" jcifs.smb.SmbException: Incorrect function. at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) at jcifs.smb.SmbTransport.send(SmbTransport.java:663) at jcifs.smb.SmbSession.sessionSetup(SmbSession.java:316) at jcifs.smb.SmbSession.send(SmbSession.java:218) at jcifs.smb.SmbTree.treeConnect(SmbTree.java:176) at jcifs.smb.SmbFile.doConnect(SmbFile.java:911) ..... *Thanks and Regards, Swapna.* * On Fri, Dec 9, 2011 at 11:46 PM, Karl Wright <daddy...@gmail.com> wrote: > Michael Allen's response to your capture: > > "It looks like the non-JCIFS communication is successful because the > anonymous credential is used. > > Note that there is still no sign of DFS going on anywhere. Actually > JCIFS tries to do a referral for the DFS root and gets an error of > STATUS_FS_DRIVER_REQUIRED so it looks like DFS may not even be setup > at all in the target environment. At least not domain based DFS. > > So if you want to emulate Windows behavior, I guess you need to retry > the operation with credential NtlmPasswordAuthentication.ANONYMOUS (or > all empty strings should work too I think like ";:")." > > In other words, the issue is that the server in question does not > actually recognize the credentials given at all, and gives an error. > But while Windows then retries with no credentials whatsoever, nothing > in JCIFS does that retry. > > You can confirm that this is what is happening by using an empty user > name, password, and domain in the JCIFS test code you wrote to obtain > the captures. If, then, the test code works against that server, I'll > create a ticket for downgrading to ANONYMOUS whenever there is an > authentication error in the JCIFS connector. > > Thanks, > Karl > > On Wed, Dec 7, 2011 at 6:41 AM, Karl Wright <daddy...@gmail.com> wrote: > > The next step is to obtain two packet captures using WireShark. These > > should both be done from the machine that is not joined to the domain. > > The first capture should be the interaction that takes place when you > > try to go to the foreign directory, and you supply the credentials, > > and then you list the folders. The second capture will require you to > > do the same using JCIFS. For that capture, you will need jcifs.jar > > and a simple example class that provides jcifs with credentials and > > tries to list the directory. A simple class should be enough to do > > this, including code such as: > > > > import jcifs.smb.*; > > > > NtlmPasswordAuthentication pa = new > > NtlmPasswordAuthentication(domain,username,password); > > SmbFile smbconnection = new SmbFile("smb://" + server + "/" + share,pa); > > SmbFile[] files = smbconnection.listFiles(); > > > > for (int i = 0 ; i < files.length ; i++) > > { > > System.out.println(file.toString()); > > } > > > > I suggest you put together your test class first and make sure it > > works, and then go after the captures. > > > > Let me know how it goes. > > > > Karl > > > > On Wed, Dec 7, 2011 at 6:10 AM, Swapna Vuppala > > <swapna.kollip...@gmail.com> wrote: > >> Hi Karl, > >> > >> Thanks for the detailed reply. > >> > >> As per your suggestion, I tried connecting to the problematic server via > >> windows using a machine that is not in the domain. On supplying the > >> credentials via windows, I could connect to the server, although it did > not > >> list any folders. But when I tried accessing the folder of my interest > via > >> \\server\folder$, I could see all the sub folders. All this is when I > >> connected via windows from a machine that's not joined to the domain. > >> > >> The behavior is same when I connect to the server via windows from a > machine > >> that's joined to the domain. > >> > >> So I assume this means that through both Kerberos and NTLM, I see the > same > >> behavior. Whereas ManifoldCF throws an exception when trying to define > the > >> repository connection. > >> > >> Please advise me on what I should be doing next to resolve this. > >> > >> Thanks and Regards, > >> Swapna. > >> > >> > >> > >> > >> On Tue, Dec 6, 2011 at 12:55 PM, Karl Wright <daddy...@gmail.com> > wrote: > >>> > >>> About your capture - Michael Allen says the following: > >>> > >>> "Actually this has nothing to do with DFS. JCIFS does not get to the > >>> point where it does DFS anything. The capture shows a vanilla > >>> STATUS_LOGON_FAILURE when GLOBAL\swapna.vuppala tries to auth with > >>> l-carx01.global.arup.com. So the possible causes for this are 1) the > >>> account name is not valid 2) the supplied password is incorrect 3) > >>> some security policy is deliberately blocking that user or particular > >>> type of auth or 4) some server configuration is incompatible with > >>> JCIFS. I only mention this last option because I noticed the target > >>> server has security signatures disabled. That's strange. If they're > >>> messing around with things like that, who knows what their clients are > >>> expected to do. > >>> > >>> Try a Windows client that uses NTLM instead of Kerberos. Meaning try a > >>> machine that is not joined to the domain so that when you try to > >>> access the target it asks you for credentials at which point you can > >>> test with GLOBAL\swapna.vuppala. Then it will use NTLM and you can > >>> actually compare captures. If the operator doesn't have a laptop or > >>> something not joined to the domain, it might be sufficient to log into > >>> a workstation using machine credentials and not domain credentials. > >>> > >>> Also when testing JCIFS you should use a simple stand-alone program > >>> like examples/ListFiles.java." > >>> > >>> In other words: > >>> (a) Since JCIFS does not use Kerberos for authentication, you need to > >>> try to log into the recalcitrant server via Windows without using > >>> Kerberos to be able to do a side-by-side comparison. Michael has some > >>> ways of doing that, above. > >>> (b) You may find that it doesn't work, in which case JCIFS is not > >>> going to work either. > >>> (c) If it *does* work, then try to generate your side-by-side > >>> comparisons using a simpler example rather than ManifoldCF en toto; > >>> you can see how at jcifs.samba.org, or I can help you further. > >>> > >>> He also mentions that there is some bizarreness on the response that > >>> indicates that the server is configured in a way that he's never seen > >>> before. And believe me, Michael has seen a *lot* of strange > >>> configurations... > >>> > >>> Hope this helps. > >>> Karl > >>> > >>> On Mon, Nov 28, 2011 at 4:12 AM, Karl Wright <daddy...@gmail.com> > wrote: > >>> > That should read "properties.xml", not "properties.ini". It looks > >>> > like this page needs updating. > >>> > > >>> > The debug property in the XML form is: > >>> > > >>> > <property name="org.apache.manifoldcf.connectors" value="DEBUG"/> > >>> > > >>> > I don't think it will provide you with any additional information > that > >>> > is useful for debugging your authentication issue, however, if that > is > >>> > why you are looking at it. There may be some jcifs.jar debugging > >>> > switches that might be of more help, but in the end I suspect you > will > >>> > need a packet capture of both a successful connection (via Windows) > >>> > and an unsuccessful one (via MCF). The guy you will need to talk > with > >>> > after that is the jcifs author Michael Allen; I can give you his > email > >>> > address if you get that far. > >>> > > >>> > Karl > >>> > > >>> > > >>> > On Mon, Nov 28, 2011 at 1:30 AM, Swapna Vuppala > >>> > <swapna.kollip...@gmail.com> wrote: > >>> >> Hi Karl, > >>> >> > >>> >> I was planning to debug jCIFS repository connection using WireShark > and > >>> >> I > >>> >> came across this > >>> >> https://cwiki.apache.org/CONNECTORS/debugging-connections.html > >>> >> Here, I see something as add > "org.apache.manifoldcf.connectors=DEBUG" > >>> >> to the > >>> >> properties.ini file. Is it the properties.xml file that is being > >>> >> referred > >>> >> here ? If not, where do I find properties.ini file ? > >>> >> > >>> >> Thanks and Regards, > >>> >> Swapna. > >>> >> > >>> >> On Thu, Nov 17, 2011 at 1:31 PM, Karl Wright <daddy...@gmail.com> > >>> >> wrote: > >>> >>> > >>> >>> See http://jcifs.samba.org/src/docs/api/overview-summary.html#scp. > >>> >>> The properties jcifs.smb.lmCompatibility and > >>> >>> jcifs.smb.client.useExtendedSecurity are the ones you may want to > >>> >>> change. These two properties go together so certain combinations > make > >>> >>> sense and others don't, so there's really only combinations you > need > >>> >>> but I'll need to look at what they are and get back to you later > >>> >>> today. > >>> >>> > >>> >>> As far as setting the switches are concerned, if you are using the > >>> >>> Quick Start you do this trivially by: > >>> >>> > >>> >>> <java> -Dxxx -Dyyy -jar start.jar > >>> >>> > >>> >>> If you are using the multi-process configuration, that is what the > >>> >>> "defines" directory is for; you only need to create files in that > >>> >>> directory with the names "jcifs.smb.lmCompatibility" and > >>> >>> "jcifs.smb.client.useExtendedSecurity" containing the values you > want > >>> >>> to set. > >>> >>> > >>> >>> Karl > >>> >>> > >>> >>> > >>> >>> On Thu, Nov 17, 2011 at 1:11 AM, Swapna Vuppala > >>> >>> <swapna.kollip...@gmail.com> wrote: > >>> >>> > Hi Karl, > >>> >>> > > >>> >>> > Am able to access the folders on the problem server through > windows > >>> >>> > explorer, (\\server3\Folder1). I tried couple of things with the > >>> >>> > credentials > >>> >>> > form, changing username, domain etc.. but I keep getting the same > >>> >>> > error > >>> >>> > "Couldn't connect to server: Logon failure: unknown user name or > bad > >>> >>> > password" > >>> >>> > > >>> >>> > Can you tell me more about the -D switch you were talking of ? > >>> >>> > > >>> >>> > Thanks and Regards, > >>> >>> > Swapna. > >>> >>> > > >>> >>> > On Tue, Nov 15, 2011 at 12:40 PM, Karl Wright < > daddy...@gmail.com> > >>> >>> > wrote: > >>> >>> >> > >>> >>> >> Glad you chased it down this far. > >>> >>> >> > >>> >>> >> First thing to try is whether you can get into the problem > server > >>> >>> >> using Windows Explorer. Obviously ManifoldCF is not going to be > >>> >>> >> able > >>> >>> >> to do it if Windows can't. If you *can* get in, then just > playing > >>> >>> >> with the form of the credentials in the MCF connection might do > the > >>> >>> >> trick. Some Windows or net appliance servers are picky about > this. > >>> >>> >> Try various things like leaving the domain blank and specifying > the > >>> >>> >> user as "a...@domain.com", for instance. There's also a > different > >>> >>> >> NTLM > >>> >>> >> mode you can operation jcifs in that some servers may be > configured > >>> >>> >> to > >>> >>> >> require; this would need you to set a -D switch on the command > line > >>> >>> >> to > >>> >>> >> enable. > >>> >>> >> > >>> >>> >> Karl > >>> >>> >> > >>> >>> >> On Tue, Nov 15, 2011 at 12:10 AM, Swapna Vuppala > >>> >>> >> <swapna.kollip...@gmail.com> wrote: > >>> >>> >> > Hi Karl, > >>> >>> >> > > >>> >>> >> > Thanks for the input. It looks like my problem is related to > the > >>> >>> >> > second > >>> >>> >> > one > >>> >>> >> > that you specified. One of the directories in the path am > trying > >>> >>> >> > to > >>> >>> >> > index is > >>> >>> >> > actually redirecting to a different server. And when I specify > >>> >>> >> > this > >>> >>> >> > new > >>> >>> >> > server in defining the repository connection, with my > >>> >>> >> > credentials, > >>> >>> >> > the > >>> >>> >> > connection fails with the message: "Couldn't connect to > server: > >>> >>> >> > Logon > >>> >>> >> > failure: unknown user name or bad password" > >>> >>> >> > > >>> >>> >> > I'll look into why am not able to connect to this server. > >>> >>> >> > > >>> >>> >> > Thanks and Regards, > >>> >>> >> > Swapna. > >>> >>> >> > > >>> >>> >> > On Mon, Nov 14, 2011 at 4:56 PM, Karl Wright < > daddy...@gmail.com> > >>> >>> >> > wrote: > >>> >>> >> >> > >>> >>> >> >> There's two kinds of problem you might be having. The first > is > >>> >>> >> >> intermittent, and the second is not intermittent but would > have > >>> >>> >> >> something to do with specific directories. > >>> >>> >> >> > >>> >>> >> >> Intermittent problems might include a domain controller that > is > >>> >>> >> >> not > >>> >>> >> >> always accessible. In such cases, the crawl will proceed but > >>> >>> >> >> will > >>> >>> >> >> tend to fail unpredictably. On the other hand, if you have a > >>> >>> >> >> directory that is handled by a DFS redirection, it is > possible > >>> >>> >> >> that > >>> >>> >> >> the redirection is indicating a new server (lets call it > >>> >>> >> >> server3) > >>> >>> >> >> which may not like the precise form of your login > credentials. > >>> >>> >> >> Can > >>> >>> >> >> you determine which scenario you are seeing? > >>> >>> >> >> > >>> >>> >> >> Karl > >>> >>> >> >> > >>> >>> >> >> On Mon, Nov 14, 2011 at 3:11 AM, Swapna Vuppala > >>> >>> >> >> <swapna.kollip...@gmail.com> wrote: > >>> >>> >> >> > Hi, > >>> >>> >> >> > > >>> >>> >> >> > I have been using windows share repository connection to > crawl > >>> >>> >> >> > and > >>> >>> >> >> > get > >>> >>> >> >> > data > >>> >>> >> >> > from a particular server (server 1). Its working perfectly > >>> >>> >> >> > fine. > >>> >>> >> >> > However, am > >>> >>> >> >> > having trouble when I try with data from another server > >>> >>> >> >> > (server > >>> >>> >> >> > 2). > >>> >>> >> >> > > >>> >>> >> >> > When I define a repository connection of type windows share > >>> >>> >> >> > and > >>> >>> >> >> > specify > >>> >>> >> >> > the > >>> >>> >> >> > server name (server 2) with my credentials, the connection > >>> >>> >> >> > status > >>> >>> >> >> > shows > >>> >>> >> >> > "Connection working". But when I run a job to use this > >>> >>> >> >> > repository > >>> >>> >> >> > connection > >>> >>> >> >> > and index data from a location on this server 2, I keep > >>> >>> >> >> > getting > >>> >>> >> >> > the > >>> >>> >> >> > exception below: > >>> >>> >> >> > > >>> >>> >> >> > JCIFS: Possibly transient exception detected on attempt 3 > >>> >>> >> >> > while > >>> >>> >> >> > checking > >>> >>> >> >> > if > >>> >>> >> >> > file exists: Logon failure: unknown user name or bad > password. > >>> >>> >> >> > jcifs.smb.SmbAuthException: Logon failure: unknown user > name > >>> >>> >> >> > or > >>> >>> >> >> > bad > >>> >>> >> >> > password. > >>> >>> >> >> > at > >>> >>> >> >> > jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:544) > >>> >>> >> >> > at jcifs.smb.SmbTransport.send(SmbTransport.java:661) > >>> >>> >> >> > at > jcifs.smb.SmbSession.sessionSetup(SmbSession.java:390) > >>> >>> >> >> > at jcifs.smb.SmbSession.send(SmbSession.java:218) > >>> >>> >> >> > at jcifs.smb.SmbTree.treeConnect(SmbTree.java:176) > >>> >>> >> >> > at jcifs.smb.SmbFile.doConnect(SmbFile.java:911) > >>> >>> >> >> > at jcifs.smb.SmbFile.connect(SmbFile.java:954) > >>> >>> >> >> > at jcifs.smb.SmbFile.connect0(SmbFile.java:880) > >>> >>> >> >> > at jcifs.smb.SmbFile.queryPath(SmbFile.java:1335) > >>> >>> >> >> > at jcifs.smb.SmbFile.exists(SmbFile.java:1417) > >>> >>> >> >> > at > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileExists(SharedDriveConnector.java:2064) > >>> >>> >> >> > at > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getDocumentVersions(SharedDriveConnector.java:521) > >>> >>> >> >> > at > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > >>> >>> >> >> > > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:318) > >>> >>> >> >> > > >>> >>> >> >> > Am able to access this location from windows explorer. What > >>> >>> >> >> > else > >>> >>> >> >> > should > >>> >>> >> >> > I be > >>> >>> >> >> > checking or what could be the reasons/factors causing this > to > >>> >>> >> >> > fail > >>> >>> >> >> > ? > >>> >>> >> >> > > >>> >>> >> >> > Thanks and Regards, > >>> >>> >> >> > Swapna. > >>> >>> >> >> > > >>> >>> >> > > >>> >>> >> > > >>> >>> > > >>> >>> > > >>> >> > >>> >> > >> > >> >