Re: Eclipse debug HDFS server side code
Thanks a lot. I will explore more related to these. On Tue, Apr 19, 2016 at 2:37 PM, Vinayakumar B <vinayakum...@apache.org> wrote: > Usually namenode console logs will be in .out file. > > -vinay > On Apr 20, 2016 12:03 AM, "Kun Ren" <ren.h...@gmail.com> wrote: > >> Hi Vinay, >> >> Thanks a lot for your quick reply. >> >> My current configuration is: I run the pseudo-distributed mode in my one >> Linux machine. >> Yes, I changed the code in eclipse first, and then re-compiled the code, >> and then restart HDFS using the new binary, and re-debug, but still can >> not see any output? When you said "the console of remote process", do you >> mean the console of the terminal that I run the command? >> >> If I want to change some code, Could you please explain a little more >> about how to debug/run my new modified code? Thanks so much. >> >> >> >> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vinayakum...@apache.org> >> wrote: >> >>> >>> -Vinay >>> >>> -- Forwarded message -- >>> From: Vinayakumar B <vinayakum...@apache.org> >>> Date: Tue, Apr 19, 2016 at 11:47 PM >>> Subject: Re: Eclipse debug HDFS server side code >>> To: Kun Ren <ren.h...@gmail.com> >>> >>> >>> 1. Since you are debugging remote code, you can't change the code >>> dynamically during debugging. If you want to change, put the compiled code >>> in remote process, and restart, reconnect and debug again. >>> 2. All the console messages will be printed on the console of remote >>> process, not in eclipse. >>> 3. Though you cannot change the code, but you can change/inspect the >>> value of value of an variable in 'display' window. >>> >>> You can explore more related to these. >>> >>> -Vinay >>> >>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <ren.h...@gmail.com> wrote: >>> >>>> Hi Vinay, >>>> >>>> Thanks a lot, it works:) >>>> >>>> I have another question: >>>> 1. When I add the following lines into the public boolean mkdirs(String >>>> src, FsPermission masked, boolean createParent) in the >>>> NameNodeRpcServer.java, >>>> >>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn"); >>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do >>>> actual work"); >>>> >>>> But I can not see any output either from my terminal or Eclipse console >>>> when I step-by-step debug this function, do you know why? >>>> >>>> 2. When I debug the client side code, for example FsShell, the default >>>> log level is INFO, I changed the level to DEBUG using the >>>> http://localhost:50070/logLevel, however, it still can not print out >>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print >>>> out debug information even I set the log level to DEBUG. >>>> >>>> >>>> Thanks a lot for your help, >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B < >>>> vinayakum...@apache.org> wrote: >>>> >>>>> Hi Kun Ren, >>>>> >>>>> You can follow the below steps. >>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug >>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in >>>>> hadoop-env.sh >>>>> 2. Start Namenode >>>>> 3. Now Namenode will start debug port in 3988. >>>>> 4. Configure Remote debug application to connect to >>>>> :3988 in eclipse >>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you >>>>> want to debug. NameNodeRpcServer.java is the gateway between NameNode and >>>>> IPC Server Layer. >>>>> 6. Now you can do step-by-step execution to observe the detailed >>>>> executions. >>>>> >>>>> Happy Debugging ;) >>>>> >>>>> -Vinay >>>>> >>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <ren.h...@gmail.com> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> Currently I used eclipse to compile/debug the source code, and I >>>>>> configured the "remote Java application" to debug the source code in >>>>>> eclipse, for example, I can debug the client side code when I run the >>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> >>>>>> DistributedFileSystem>DFSClient ->send request to the server >>>>>> side. >>>>>> Then it finish debugging, but I can not see how the server side handle >>>>>> the >>>>>> request. >>>>>> >>>>>> >>>>>> However, I want to debug and see how the server side(HDFS) handle the >>>>>> request? Do you know how I can do this? Thanks so much. Also when I try >>>>>> to >>>>>> add some debug information(such as print out something), but I can not >>>>>> see >>>>>> anything printed out in eclipse, I don't know why. Thanks again. >>>>>> >>>>>> Best, >>>>>> Kun >>>>>> >>>>> >>>>> >>>> >>> >>> >>
Re: Eclipse debug HDFS server side code
Usually namenode console logs will be in .out file. -vinay On Apr 20, 2016 12:03 AM, "Kun Ren" <ren.h...@gmail.com> wrote: > Hi Vinay, > > Thanks a lot for your quick reply. > > My current configuration is: I run the pseudo-distributed mode in my one > Linux machine. > Yes, I changed the code in eclipse first, and then re-compiled the code, > and then restart HDFS using the new binary, and re-debug, but still can > not see any output? When you said "the console of remote process", do you > mean the console of the terminal that I run the command? > > If I want to change some code, Could you please explain a little more > about how to debug/run my new modified code? Thanks so much. > > > > On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vinayakum...@apache.org> > wrote: > >> >> -Vinay >> >> -- Forwarded message ------ >> From: Vinayakumar B <vinayakum...@apache.org> >> Date: Tue, Apr 19, 2016 at 11:47 PM >> Subject: Re: Eclipse debug HDFS server side code >> To: Kun Ren <ren.h...@gmail.com> >> >> >> 1. Since you are debugging remote code, you can't change the code >> dynamically during debugging. If you want to change, put the compiled code >> in remote process, and restart, reconnect and debug again. >> 2. All the console messages will be printed on the console of remote >> process, not in eclipse. >> 3. Though you cannot change the code, but you can change/inspect the >> value of value of an variable in 'display' window. >> >> You can explore more related to these. >> >> -Vinay >> >> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <ren.h...@gmail.com> wrote: >> >>> Hi Vinay, >>> >>> Thanks a lot, it works:) >>> >>> I have another question: >>> 1. When I add the following lines into the public boolean mkdirs(String >>> src, FsPermission masked, boolean createParent) in the >>> NameNodeRpcServer.java, >>> >>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn"); >>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do >>> actual work"); >>> >>> But I can not see any output either from my terminal or Eclipse console >>> when I step-by-step debug this function, do you know why? >>> >>> 2. When I debug the client side code, for example FsShell, the default >>> log level is INFO, I changed the level to DEBUG using the >>> http://localhost:50070/logLevel, however, it still can not print out >>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print >>> out debug information even I set the log level to DEBUG. >>> >>> >>> Thanks a lot for your help, >>> >>> >>> >>> >>> >>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vinayakum...@apache.org >>> > wrote: >>> >>>> Hi Kun Ren, >>>> >>>> You can follow the below steps. >>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug >>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in >>>> hadoop-env.sh >>>> 2. Start Namenode >>>> 3. Now Namenode will start debug port in 3988. >>>> 4. Configure Remote debug application to connect to >>>> :3988 in eclipse >>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want >>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC >>>> Server Layer. >>>> 6. Now you can do step-by-step execution to observe the detailed >>>> executions. >>>> >>>> Happy Debugging ;) >>>> >>>> -Vinay >>>> >>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <ren.h...@gmail.com> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Currently I used eclipse to compile/debug the source code, and I >>>>> configured the "remote Java application" to debug the source code in >>>>> eclipse, for example, I can debug the client side code when I run the >>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> >>>>> DistributedFileSystem>DFSClient ->send request to the server side. >>>>> Then it finish debugging, but I can not see how the server side handle the >>>>> request. >>>>> >>>>> >>>>> However, I want to debug and see how the server side(HDFS) handle the >>>>> request? Do you know how I can do this? Thanks so much. Also when I try >>>>> to >>>>> add some debug information(such as print out something), but I can not see >>>>> anything printed out in eclipse, I don't know why. Thanks again. >>>>> >>>>> Best, >>>>> Kun >>>>> >>>> >>>> >>> >> >> >
Re: Eclipse debug HDFS server side code
Hi Vinay, Thanks a lot for your quick reply. My current configuration is: I run the pseudo-distributed mode in my one Linux machine. Yes, I changed the code in eclipse first, and then re-compiled the code, and then restart HDFS using the new binary, and re-debug, but still can not see any output? When you said "the console of remote process", do you mean the console of the terminal that I run the command? If I want to change some code, Could you please explain a little more about how to debug/run my new modified code? Thanks so much. On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vinayakum...@apache.org> wrote: > > -Vinay > > -- Forwarded message -- > From: Vinayakumar B <vinayakum...@apache.org> > Date: Tue, Apr 19, 2016 at 11:47 PM > Subject: Re: Eclipse debug HDFS server side code > To: Kun Ren <ren.h...@gmail.com> > > > 1. Since you are debugging remote code, you can't change the code > dynamically during debugging. If you want to change, put the compiled code > in remote process, and restart, reconnect and debug again. > 2. All the console messages will be printed on the console of remote > process, not in eclipse. > 3. Though you cannot change the code, but you can change/inspect the value > of value of an variable in 'display' window. > > You can explore more related to these. > > -Vinay > > On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <ren.h...@gmail.com> wrote: > >> Hi Vinay, >> >> Thanks a lot, it works:) >> >> I have another question: >> 1. When I add the following lines into the public boolean mkdirs(String >> src, FsPermission masked, boolean createParent) in the >> NameNodeRpcServer.java, >> >> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn"); >> System.out.println("In NameNodeRpcServer.java:mkdirs, before do >> actual work"); >> >> But I can not see any output either from my terminal or Eclipse console >> when I step-by-step debug this function, do you know why? >> >> 2. When I debug the client side code, for example FsShell, the default >> log level is INFO, I changed the level to DEBUG using the >> http://localhost:50070/logLevel, however, it still can not print out >> LOG.debug(), but I can see LOG.info(), I don't know why it can not print >> out debug information even I set the log level to DEBUG. >> >> >> Thanks a lot for your help, >> >> >> >> >> >> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vinayakum...@apache.org> >> wrote: >> >>> Hi Kun Ren, >>> >>> You can follow the below steps. >>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug >>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in >>> hadoop-env.sh >>> 2. Start Namenode >>> 3. Now Namenode will start debug port in 3988. >>> 4. Configure Remote debug application to connect to :3988 >>> in eclipse >>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want >>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC >>> Server Layer. >>> 6. Now you can do step-by-step execution to observe the detailed >>> executions. >>> >>> Happy Debugging ;) >>> >>> -Vinay >>> >>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <ren.h...@gmail.com> wrote: >>> >>>> Hi All, >>>> >>>> Currently I used eclipse to compile/debug the source code, and I >>>> configured the "remote Java application" to debug the source code in >>>> eclipse, for example, I can debug the client side code when I run the >>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> >>>> DistributedFileSystem>DFSClient ->send request to the server side. >>>> Then it finish debugging, but I can not see how the server side handle the >>>> request. >>>> >>>> >>>> However, I want to debug and see how the server side(HDFS) handle the >>>> request? Do you know how I can do this? Thanks so much. Also when I try to >>>> add some debug information(such as print out something), but I can not see >>>> anything printed out in eclipse, I don't know why. Thanks again. >>>> >>>> Best, >>>> Kun >>>> >>> >>> >> > >
Fwd: Eclipse debug HDFS server side code
-Vinay -- Forwarded message -- From: Vinayakumar B <vinayakum...@apache.org> Date: Tue, Apr 19, 2016 at 11:47 PM Subject: Re: Eclipse debug HDFS server side code To: Kun Ren <ren.h...@gmail.com> 1. Since you are debugging remote code, you can't change the code dynamically during debugging. If you want to change, put the compiled code in remote process, and restart, reconnect and debug again. 2. All the console messages will be printed on the console of remote process, not in eclipse. 3. Though you cannot change the code, but you can change/inspect the value of value of an variable in 'display' window. You can explore more related to these. -Vinay On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <ren.h...@gmail.com> wrote: > Hi Vinay, > > Thanks a lot, it works:) > > I have another question: > 1. When I add the following lines into the public boolean mkdirs(String > src, FsPermission masked, boolean createParent) in the > NameNodeRpcServer.java, > > LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn"); > System.out.println("In NameNodeRpcServer.java:mkdirs, before do > actual work"); > > But I can not see any output either from my terminal or Eclipse console > when I step-by-step debug this function, do you know why? > > 2. When I debug the client side code, for example FsShell, the default > log level is INFO, I changed the level to DEBUG using the > http://localhost:50070/logLevel, however, it still can not print out > LOG.debug(), but I can see LOG.info(), I don't know why it can not print > out debug information even I set the log level to DEBUG. > > > Thanks a lot for your help, > > > > > > On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vinayakum...@apache.org> > wrote: > >> Hi Kun Ren, >> >> You can follow the below steps. >> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug >> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in >> hadoop-env.sh >> 2. Start Namenode >> 3. Now Namenode will start debug port in 3988. >> 4. Configure Remote debug application to connect to :3988 >> in eclipse >> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want >> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC >> Server Layer. >> 6. Now you can do step-by-step execution to observe the detailed >> executions. >> >> Happy Debugging ;) >> >> -Vinay >> >> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <ren.h...@gmail.com> wrote: >> >>> Hi All, >>> >>> Currently I used eclipse to compile/debug the source code, and I >>> configured the "remote Java application" to debug the source code in >>> eclipse, for example, I can debug the client side code when I run the >>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> >>> DistributedFileSystem>DFSClient ->send request to the server side. >>> Then it finish debugging, but I can not see how the server side handle the >>> request. >>> >>> >>> However, I want to debug and see how the server side(HDFS) handle the >>> request? Do you know how I can do this? Thanks so much. Also when I try to >>> add some debug information(such as print out something), but I can not see >>> anything printed out in eclipse, I don't know why. Thanks again. >>> >>> Best, >>> Kun >>> >> >> >
Re: Eclipse debug HDFS server side code
Hi Vinay, Thanks a lot, it works:) I have another question: 1. When I add the following lines into the public boolean mkdirs(String src, FsPermission masked, boolean createParent) in the NameNodeRpcServer.java, LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn"); System.out.println("In NameNodeRpcServer.java:mkdirs, before do actual work"); But I can not see any output either from my terminal or Eclipse console when I step-by-step debug this function, do you know why? 2. When I debug the client side code, for example FsShell, the default log level is INFO, I changed the level to DEBUG using the http://localhost:50070/logLevel, however, it still can not print out LOG.debug(), but I can see LOG.info(), I don't know why it can not print out debug information even I set the log level to DEBUG. Thanks a lot for your help, On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar Bwrote: > Hi Kun Ren, > > You can follow the below steps. > 1. configure HADOOP_NAMENODE_OPTS="-Xdebug > -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in > hadoop-env.sh > 2. Start Namenode > 3. Now Namenode will start debug port in 3988. > 4. Configure Remote debug application to connect to :3988 > in eclipse > 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to > debug. NameNodeRpcServer.java is the gateway between NameNode and IPC > Server Layer. > 6. Now you can do step-by-step execution to observe the detailed > executions. > > Happy Debugging ;) > > -Vinay > > On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren wrote: > >> Hi All, >> >> Currently I used eclipse to compile/debug the source code, and I >> configured the "remote Java application" to debug the source code in >> eclipse, for example, I can debug the client side code when I run the >> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> >> DistributedFileSystem>DFSClient ->send request to the server side. >> Then it finish debugging, but I can not see how the server side handle the >> request. >> >> >> However, I want to debug and see how the server side(HDFS) handle the >> request? Do you know how I can do this? Thanks so much. Also when I try to >> add some debug information(such as print out something), but I can not see >> anything printed out in eclipse, I don't know why. Thanks again. >> >> Best, >> Kun >> > >
Re: Eclipse debug HDFS server side code
Hi Kun Ren, You can follow the below steps. 1. configure HADOOP_NAMENODE_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in hadoop-env.sh 2. Start Namenode 3. Now Namenode will start debug port in 3988. 4. Configure Remote debug application to connect to :3988 in eclipse 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC Server Layer. 6. Now you can do step-by-step execution to observe the detailed executions. Happy Debugging ;) -Vinay On Tue, Apr 19, 2016 at 8:46 PM, Kun Renwrote: > Hi All, > > Currently I used eclipse to compile/debug the source code, and I > configured the "remote Java application" to debug the source code in > eclipse, for example, I can debug the client side code when I run the > command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> > DistributedFileSystem>DFSClient ->send request to the server side. > Then it finish debugging, but I can not see how the server side handle the > request. > > > However, I want to debug and see how the server side(HDFS) handle the > request? Do you know how I can do this? Thanks so much. Also when I try to > add some debug information(such as print out something), but I can not see > anything printed out in eclipse, I don't know why. Thanks again. > > Best, > Kun >
Eclipse debug HDFS server side code
Hi All, Currently I used eclipse to compile/debug the source code, and I configured the "remote Java application" to debug the source code in eclipse, for example, I can debug the client side code when I run the command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> DistributedFileSystem>DFSClient ->send request to the server side. Then it finish debugging, but I can not see how the server side handle the request. However, I want to debug and see how the server side(HDFS) handle the request? Do you know how I can do this? Thanks so much. Also when I try to add some debug information(such as print out something), but I can not see anything printed out in eclipse, I don't know why. Thanks again. Best, Kun