I've having some problems creating a new file on HDFS. I am attempting to do this after my MapReduce job has finished and I am trying to combine all part-00* files into a single file programmatically. It's throwing a LeaseExpiredException saying the file I just created doesn't exist. Any idea why this is happening or what I can do to fix it? -Xavier Here is the code snippet ======================================================================== ======================================= FileSystem fileSys = FileSystem.get(job); FSDataOutputStream fsdos = fileSys.create(new Path(outputIndexPath)); if (!fileSys.exists(new Path(outputIndexPath))) { System.err.println("File still does not exist: "+outputIndexPath); } else { System.out.println("File exists: "+outputIndexPath); } BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(fsdos,"UTF-8"));
Output with stack trace ======================================================================== ======================================= File exists: output/index.txt 08/05/14 03:20:13 INFO dfs.DFSClient: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.LeaseExpiredException: No lease on /user/xstevens/output/index.txt File does not exist. [Lease. Holder: 44 46 53 43 6c 69 65 6e 74 5f 2d 31 30 31 34 35 38 35 32 32 33, heldlocks: 0, pendingcreates: 1] at org.apache.hadoop.dfs.FSNamesystem.checkLease(FSNamesystem.java:1160) at org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java: 1097) at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:312) at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor Impl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:901) at org.apache.hadoop.ipc.Client.call(Client.java:512) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198) at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav a:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor Impl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvo cationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocation Handler.java:59) at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source) at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFS Client.java:2074) at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DF SClient.java:1967) at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1500(DFSClient.ja va:1487) at org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClie nt.java:1601) Xavier stevens Sr. software engineer FOX INTERACTIVE MEDIA e: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> p: 310.633.9749