Re: Establishment of Static Source Code Analysis

2017-06-16 Thread Julien Nioche 
>
>  Russian compatriots


Are we all Russian then?

On 16 June 2017 at 04:29, lewis john mcgibbney  wrote:

> Hi Folks,
> I don't know if anyone else noticed... some of our Russian compatriots
> have set up a static auto bot to notify us of source code issues...
> An example is as follows
> https://issues.apache.org/jira/browse/NUTCH-2394
> I think this is great to be honest... with some peer review I think we
> could take this seriously.
> Out of curiosity is anyone responsible for this?
> Lewis
>
> --
> http://home.apache.org/~lewismc/
> @hectorMcSpector
> http://www.linkedin.com/in/lmcgibbney
>



-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble

Re: Establishment of Static Source Code Analysis

2017-06-16 Thread Julien Nioche 
More seriously, no idea who's done it but it is useful feedback. A similar
company (DevFactory)  contributed to StormCrawler
 some
time ago. Also reminds me of the discussion we had around Sonar in
crawler-commons
.

On 16 June 2017 at 08:55, Julien Nioche 
wrote:

>  Russian compatriots
>
>
> Are we all Russian then?
>
> On 16 June 2017 at 04:29, lewis john mcgibbney  wrote:
>
>> Hi Folks,
>> I don't know if anyone else noticed... some of our Russian compatriots
>> have set up a static auto bot to notify us of source code issues...
>> An example is as follows
>> https://issues.apache.org/jira/browse/NUTCH-2394
>> I think this is great to be honest... with some peer review I think we
>> could take this seriously.
>> Out of curiosity is anyone responsible for this?
>> Lewis
>>
>> --
>> http://home.apache.org/~lewismc/
>> @hectorMcSpector
>> http://www.linkedin.com/in/lmcgibbney
>>
>
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 
>



-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble

Re: Establishment of Static Source Code Analysis

2017-06-16 Thread Furkan KAMACI 
Such bugs and vulnerabilities can be detected by Sonar (NUTCH-1109) as you
know. This is not accessible:
https://svn.apache.org/repos/asf/nutch/site/publish/sonar.html#Sonar+Analysis
it
could be nice to check them periodically via a dashboard.


On Fri, Jun 16, 2017 at 12:35 PM, Julien Nioche <
lists.digitalpeb...@gmail.com> wrote:

> More seriously, no idea who's done it but it is useful feedback. A similar
> company (DevFactory)  contributed to StormCrawler
>  some
> time ago. Also reminds me of the discussion we had around Sonar in
> crawler-commons
> .
>
> On 16 June 2017 at 08:55, Julien Nioche 
> wrote:
>
>>  Russian compatriots
>>
>>
>> Are we all Russian then?
>>
>> On 16 June 2017 at 04:29, lewis john mcgibbney 
>> wrote:
>>
>>> Hi Folks,
>>> I don't know if anyone else noticed... some of our Russian compatriots
>>> have set up a static auto bot to notify us of source code issues...
>>> An example is as follows
>>> https://issues.apache.org/jira/browse/NUTCH-2394
>>> I think this is great to be honest... with some peer review I think we
>>> could take this seriously.
>>> Out of curiosity is anyone responsible for this?
>>> Lewis
>>>
>>> --
>>> http://home.apache.org/~lewismc/
>>> @hectorMcSpector
>>> http://www.linkedin.com/in/lmcgibbney
>>>
>>
>>
>>
>> --
>>
>> *Open Source Solutions for Text Engineering*
>>
>> http://www.digitalpebble.com
>> http://digitalpebble.blogspot.com/
>> #digitalpebble 
>>
>
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 
>

[jira] [Commented] (NUTCH-2375) Upgrade the code base from org.apache.hadoop.mapred to org.apache.hadoop.mapreduce

2017-06-16 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NUTCH-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051895#comment-16051895
 ] 

ASF GitHub Bot commented on NUTCH-2375:
---

lewismc commented on issue #188: NUTCH-2375 Upgrade the code base from 
org.apache.hadoop.mapred to org.apache.hadoop.mapreduce
URL: https://github.com/apache/nutch/pull/188#issuecomment-309027522
 
 
   Hi @Omkar20895 I am getting the following build errors
   ```
   compile-core:
   [javac] Compiling 284 source files to /usr/local/nutch/build/classes
   [javac] 
/usr/local/nutch/src/java/org/apache/nutch/crawl/CrawlDb.java:119: error: 
unreported exception InterruptedException; must be caught or declared to be 
thrown
   [javac]   int complete = job.waitForCompletion(true)?0:1;
   [javac]   ^
   [javac] 
/usr/local/nutch/src/java/org/apache/nutch/crawl/CrawlDbReader.java:399: error: 
cannot find symbol
   [javac]SequenceFile.Reader[] readers = 
SequenceFileOutputFormat.getReaders(config,
   [javac]^
   [javac]   symbol:   method getReaders(Configuration,Path)
   [javac]   location: class SequenceFileOutputFormat
   [javac] 
/usr/local/nutch/src/java/org/apache/nutch/hostdb/ReadHostDb.java:208: error: 
cannot find symbol
   [javac] SequenceFile.Reader[] readers = 
SequenceFileOutputFormat.getReaders(conf, hostDb);
   [javac] ^
   [javac]   symbol:   method getReaders(Configuration,Path)
   [javac]   location: class SequenceFileOutputFormat
   [javac] 
/usr/local/nutch/src/java/org/apache/nutch/segment/SegmentReader.java:434: 
error: cannot find symbol
   [javac] SequenceFile.Reader[] readers = 
SequenceFileOutputFormat.getReaders(
   [javac] ^
   [javac]   symbol:   method getReaders(Configuration,Path)
   [javac]   location: class SequenceFileOutputFormat
   [javac] 
/usr/local/nutch/src/java/org/apache/nutch/segment/SegmentReader.java:511: 
error: cannot find symbol
   [javac]   SequenceFile.Reader[] readers = 
SequenceFileOutputFormat.getReaders(
   [javac]   ^
   [javac]   symbol:   method getReaders(Configuration,Path)
   [javac]   location: class SequenceFileOutputFormat
   [javac] 
/usr/local/nutch/src/java/org/apache/nutch/tools/FreeGenerator.java:197: error: 
cannot find symbol
   [javac] 
job.setOutputKeyComparatorClass(Generator.HashComparator.class);
   [javac]^
   [javac]   symbol:   method 
setOutputKeyComparatorClass(Class)
   [javac]   location: variable job of type Job
   [javac] 
/usr/local/nutch/src/java/org/apache/nutch/util/NutchJob.java:29: warning: 
[deprecation] Job(Configuration,String) in Job has been deprecated
   [javac] super(conf, "NutchJob");
   [javac] ^
   [javac] Note: Some input files use unchecked or unsafe operations.
   [javac] Note: Recompile with -Xlint:unchecked for details.
   [javac] 6 errors
   [javac] 1 warning
   ```
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Upgrade the code base from org.apache.hadoop.mapred to 
> org.apache.hadoop.mapreduce
> --
>
> Key: NUTCH-2375
> URL: https://issues.apache.org/jira/browse/NUTCH-2375
> Project: Nutch
>  Issue Type: Improvement
>  Components: deployment
>Reporter: Omkar Reddy
>
> Nutch is still using the deprecated org.apache.hadoop.mapred dependency which 
> has been deprecated. It need to be updated to org.apache.hadoop.mapreduce 
> dependency. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NUTCH-2375) Upgrade the code base from org.apache.hadoop.mapred to org.apache.hadoop.mapreduce

2017-06-16 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NUTCH-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051898#comment-16051898
 ] 

ASF GitHub Bot commented on NUTCH-2375:
---

lewismc commented on issue #188: NUTCH-2375 Upgrade the code base from 
org.apache.hadoop.mapred to org.apache.hadoop.mapreduce
URL: https://github.com/apache/nutch/pull/188#issuecomment-309027722
 
 
   Please start with the following issue
   ```
   [javac]   symbol:   method 
setOutputKeyComparatorClass(Class)
   [javac]   location: variable job of type Job
   [javac] 
/usr/local/nutch/src/java/org/apache/nutch/util/NutchJob.java:29: warning: 
[deprecation] Job(Configuration,String) in Job has been deprecated
   [javac] super(conf, "NutchJob");
   [javac] ^
   ```
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Upgrade the code base from org.apache.hadoop.mapred to 
> org.apache.hadoop.mapreduce
> --
>
> Key: NUTCH-2375
> URL: https://issues.apache.org/jira/browse/NUTCH-2375
> Project: Nutch
>  Issue Type: Improvement
>  Components: deployment
>Reporter: Omkar Reddy
>
> Nutch is still using the deprecated org.apache.hadoop.mapred dependency which 
> has been deprecated. It need to be updated to org.apache.hadoop.mapreduce 
> dependency. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NUTCH-2375) Upgrade the code base from org.apache.hadoop.mapred to org.apache.hadoop.mapreduce

2017-06-16 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NUTCH-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051906#comment-16051906
 ] 

ASF GitHub Bot commented on NUTCH-2375:
---

lewismc commented on issue #188: NUTCH-2375 Upgrade the code base from 
org.apache.hadoop.mapred to org.apache.hadoop.mapreduce
URL: https://github.com/apache/nutch/pull/188#issuecomment-309028312
 
 
   @Omkar20895 please refer to following class for guidance
   
https://github.com/apache/nutch/blob/2.x/src/java/org/apache/nutch/util/NutchJob.java
   Once you push a commit please ping me. Thank you.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Upgrade the code base from org.apache.hadoop.mapred to 
> org.apache.hadoop.mapreduce
> --
>
> Key: NUTCH-2375
> URL: https://issues.apache.org/jira/browse/NUTCH-2375
> Project: Nutch
>  Issue Type: Improvement
>  Components: deployment
>Reporter: Omkar Reddy
>
> Nutch is still using the deprecated org.apache.hadoop.mapred dependency which 
> has been deprecated. It need to be updated to org.apache.hadoop.mapreduce 
> dependency. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)