Re: Build failed in Jenkins: ManifoldCF-mvn #722

2019-09-03 Thread Karl Wright 
I just added you -- I think.  Please check.
Karl


On Tue, Sep 3, 2019 at 4:52 PM Markus Schuch  wrote:

> Ok, i will open a infra ticket.
>
> about jenkins permissions:
>
> I can login with my apache username but i have noch access to job
> configurations.
>
> According to https://cwiki.apache.org/confluence/display/INFRA/Jenkins
> PMC chairs can grant access through whimsy at
> https://whimsy.apache.org/roster/group/hudson-jobadmin
>
> Markus
>
>
> Am 27.08.2019 um 09:56 schrieb Karl Wright:
> > I see that maven-3.2.1 is specified in the configuration, and that
> > furthermore it hasn't changed for a long time.  I think therefore we're
> > seeing an infrastructure problem.   Maybe open an infra ticket?
> > Karl
> >
> > On Tue, Aug 27, 2019 at 3:52 AM Karl Wright  wrote:
> >
> >> I'm note even sure how jenkins permissions are granted at Apache.  I
> just
> >> log in using my apache username and password and Jenkins privs have been
> >> always granted.  What do you see?
> >>
> >> Karl
> >>
> >> On Tue, Aug 27, 2019 at 1:45 AM Markus Schuch 
> >> wrote:
> >>
> >>> Error: FATAL: Couldn't find any executable in
> >>> /home/jenkins/tools/maven/apache-maven-3.2.1
> >>>
> >>> I'am not able to relate that to my recent commits.
> >>>
> >>> According to
> >>>
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Matrix
> Maven
> >>> 3.2.1 should still be installed on the slaves.
> >>>
> >>> The path looks correct: /home/jenkins/tools/maven/apache-maven-3.2.1
> >>>
> >>> Configuration access for our jobs in jenkins might help to unterstand
> >>> this better.
> >>>
> >>> Am 27. August 2019 07:41:17 MESZ schrieb Apache Jenkins Server <
> >>> jenk...@builds.apache.org>:
>  See <
> https://builds.apache.org/job/ManifoldCF-mvn/722/display/redirect>
> 
>  --
>  [...truncated 435.83 KB...]
>  A connectors/nulltransformation
>  A connectors/nulltransformation/.gitignore
>  A connectors/nulltransformation/connector
>  A connectors/nulltransformation/connector/src
>  A connectors/nulltransformation/connector/src/main
>  A connectors/nulltransformation/connector/src/main/java
>  A connectors/nulltransformation/connector/src/main/java/org
>  A
>  connectors/nulltransformation/connector/src/main/java/org/apache
>  A
> >>>
> 
> connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf
>  A
> >>>
> 
> connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf/agents
>  A
> >>>
> 
> connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf/agents/transformation
>  A
> >>>
> 
> connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf/agents/transformation/nullconnector
>  AU
> >>>
> 
> connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf/agents/transformation/nullconnector/NullConnector.java
>  AUconnectors/nulltransformation/pom.xml
>  AUconnectors/nulltransformation/build.xml
>  A connectors/amazoncloudsearch
>  A connectors/amazoncloudsearch/.gitignore
>  A connectors/amazoncloudsearch/connector
>  A connectors/amazoncloudsearch/connector/src
>  A connectors/amazoncloudsearch/connector/src/test
>  A connectors/amazoncloudsearch/connector/src/test/java
>  A connectors/amazoncloudsearch/connector/src/test/java/org
>  A
>  connectors/amazoncloudsearch/connector/src/test/java/org/apache
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf/agents
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf/agents/output
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf/agents/output/amazoncloudsearch
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf/agents/output/amazoncloudsearch/tests
>  A connectors/amazoncloudsearch/connector/src/main
>  A connectors/amazoncloudsearch/connector/src/main/resources
>  A
>  connectors/amazoncloudsearch/connector/src/main/resources/org
>  A
>  connectors/amazoncloudsearch/connector/src/main/resources/org/apache
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf/agents
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf/agents/output
>  A
> >>>
> 
> connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf/agents/output/amazoncloudsearch
>  AU

[jira] [Commented] (CONNECTORS-1105) Add maven delivery targets to poms

2019-09-03 Thread Markus Schuch (Jira) 


[ 
https://issues.apache.org/jira/browse/CONNECTORS-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921725#comment-16921725
 ] 

Markus Schuch commented on CONNECTORS-1105:
---

I do not have edit permissions in the wiki space. [~kwri...@metacarta.com] are 
you able to add me to the {{manifoldcf-committers}}-group?

> Add maven delivery targets to poms
> --
>
> Key: CONNECTORS-1105
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1105
> Project: ManifoldCF
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: ManifoldCF 1.8
>Reporter: Karl Wright
>Assignee: Markus Schuch
>Priority: Major
> Fix For: ManifoldCF next
>
>
> We've been asked to deliver mcf jars and wars to maven central repository by 
> some developers.  This ticket represents that work.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (CONNECTORS-1105) Add maven delivery targets to poms

2019-09-03 Thread Markus Schuch (Jira) 


[ 
https://issues.apache.org/jira/browse/CONNECTORS-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921716#comment-16921716
 ] 

Markus Schuch commented on CONNECTORS-1105:
---

The referred wiki pages are
 * [https://cwiki.apache.org/confluence/display/CONNECTORS/Release+Process]
 * 
[https://cwiki.apache.org/confluence/display/CONNECTORS/Distributing+Releases]

I will write up the steps there.

 

> Add maven delivery targets to poms
> --
>
> Key: CONNECTORS-1105
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1105
> Project: ManifoldCF
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: ManifoldCF 1.8
>Reporter: Karl Wright
>Assignee: Markus Schuch
>Priority: Major
> Fix For: ManifoldCF next
>
>
> We've been asked to deliver mcf jars and wars to maven central repository by 
> some developers.  This ticket represents that work.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (CONNECTORS-1105) Add maven delivery targets to poms

2019-09-03 Thread Markus Schuch (Jira) 


[ 
https://issues.apache.org/jira/browse/CONNECTORS-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921716#comment-16921716
 ] 

Markus Schuch edited comment on CONNECTORS-1105 at 9/3/19 8:56 PM:
---

The referred wiki pages are
 * [https://cwiki.apache.org/confluence/display/CONNECTORS/Release+Process]
 * 
[https://cwiki.apache.org/confluence/display/CONNECTORS/Distributing+Releases]

I will write up the steps there.


was (Author: schuchm):
The referred wiki pages are
 * [https://cwiki.apache.org/confluence/display/CONNECTORS/Release+Process]
 * 
[https://cwiki.apache.org/confluence/display/CONNECTORS/Distributing+Releases]

I will write up the steps there.

 

> Add maven delivery targets to poms
> --
>
> Key: CONNECTORS-1105
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1105
> Project: ManifoldCF
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: ManifoldCF 1.8
>Reporter: Karl Wright
>Assignee: Markus Schuch
>Priority: Major
> Fix For: ManifoldCF next
>
>
> We've been asked to deliver mcf jars and wars to maven central repository by 
> some developers.  This ticket represents that work.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: Build failed in Jenkins: ManifoldCF-mvn #722

2019-09-03 Thread Markus Schuch 
Ok, i will open a infra ticket.

about jenkins permissions:

I can login with my apache username but i have noch access to job
configurations.

According to https://cwiki.apache.org/confluence/display/INFRA/Jenkins
PMC chairs can grant access through whimsy at
https://whimsy.apache.org/roster/group/hudson-jobadmin

Markus


Am 27.08.2019 um 09:56 schrieb Karl Wright:
> I see that maven-3.2.1 is specified in the configuration, and that
> furthermore it hasn't changed for a long time.  I think therefore we're
> seeing an infrastructure problem.   Maybe open an infra ticket?
> Karl
>
> On Tue, Aug 27, 2019 at 3:52 AM Karl Wright  wrote:
>
>> I'm note even sure how jenkins permissions are granted at Apache.  I just
>> log in using my apache username and password and Jenkins privs have been
>> always granted.  What do you see?
>>
>> Karl
>>
>> On Tue, Aug 27, 2019 at 1:45 AM Markus Schuch 
>> wrote:
>>
>>> Error: FATAL: Couldn't find any executable in
>>> /home/jenkins/tools/maven/apache-maven-3.2.1
>>>
>>> I'am not able to relate that to my recent commits.
>>>
>>> According to
>>> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Matrix 
>>> Maven
>>> 3.2.1 should still be installed on the slaves.
>>>
>>> The path looks correct: /home/jenkins/tools/maven/apache-maven-3.2.1
>>>
>>> Configuration access for our jobs in jenkins might help to unterstand
>>> this better.
>>>
>>> Am 27. August 2019 07:41:17 MESZ schrieb Apache Jenkins Server <
>>> jenk...@builds.apache.org>:
 See 

 --
 [...truncated 435.83 KB...]
 A connectors/nulltransformation
 A connectors/nulltransformation/.gitignore
 A connectors/nulltransformation/connector
 A connectors/nulltransformation/connector/src
 A connectors/nulltransformation/connector/src/main
 A connectors/nulltransformation/connector/src/main/java
 A connectors/nulltransformation/connector/src/main/java/org
 A
 connectors/nulltransformation/connector/src/main/java/org/apache
 A
>>>
 connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf
 A
>>>
 connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf/agents
 A
>>>
 connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf/agents/transformation
 A
>>>
 connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf/agents/transformation/nullconnector
 AU
>>>
 connectors/nulltransformation/connector/src/main/java/org/apache/manifoldcf/agents/transformation/nullconnector/NullConnector.java
 AUconnectors/nulltransformation/pom.xml
 AUconnectors/nulltransformation/build.xml
 A connectors/amazoncloudsearch
 A connectors/amazoncloudsearch/.gitignore
 A connectors/amazoncloudsearch/connector
 A connectors/amazoncloudsearch/connector/src
 A connectors/amazoncloudsearch/connector/src/test
 A connectors/amazoncloudsearch/connector/src/test/java
 A connectors/amazoncloudsearch/connector/src/test/java/org
 A
 connectors/amazoncloudsearch/connector/src/test/java/org/apache
 A
>>>
 connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf
 A
>>>
 connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf/agents
 A
>>>
 connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf/agents/output
 A
>>>
 connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf/agents/output/amazoncloudsearch
 A
>>>
 connectors/amazoncloudsearch/connector/src/test/java/org/apache/manifoldcf/agents/output/amazoncloudsearch/tests
 A connectors/amazoncloudsearch/connector/src/main
 A connectors/amazoncloudsearch/connector/src/main/resources
 A connectors/amazoncloudsearch/connector/src/main/resources/org
 A
 connectors/amazoncloudsearch/connector/src/main/resources/org/apache
 A
>>>
 connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf
 A
>>>
 connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf/agents
 A
>>>
 connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf/agents/output
 A
>>>
 connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf/agents/output/amazoncloudsearch
 AU
>>>
 connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf/agents/output/amazoncloudsearch/editConfiguration.js
 AU
>>>
 connectors/amazoncloudsearch/connector/src/main/resources/org/apache/manifoldcf/agents/output/amazoncloudsearch/editConfiguration.html
 AU
>>>

Re: Manifold CF-Non existent of URL

2019-09-03 Thread Karl Wright 
Yes, if mcf receives a 404 response it will delete the document from the
index.

Continuous crawling though means the document may not be retried for a long
time.  Exponential back off is used.

Karl

On Tue, Sep 3, 2019, 1:36 AM Priya Arora  wrote:

> Yes its a  continuous   Job.
>
> On Tue, Sep 3, 2019 at 11:05 AM Priya Arora  wrote:
>
> > Hi ,
> > I am having a job Job:-myuniversity_intranet (which is crawling data from
> > intranet site) and the data has been indexed in an index.
> > My query here is, does manifold have some functionality to test a url
> > before indexing that whether the URL is existing or not?.
> > Likewise , in my index (say index name: abc), i am having URL(indexed).
> > URL:- https:myuniversity/reaserch/info(which is an intranet url). This
> URL
> > was existing earlier but not existing now, and resulting status is 404.
> >
> > Query is :- Can monifoldcf checks before indexing whether its status is
> > not equal to 404(that means it exists). if the URL exists in real only
> then
> > index otherwise skip that URL.
> > Does this setting can be implemented while configuring manifold cf job.,
> > or do I have to manually handle this in code.
> >
> >
> > Kind regards
> > Priya
> >
> > On Mon, Sep 2, 2019 at 8:19 PM Karl Wright  wrote:
> >
> >> Hi,
> >> You aren't giving me enough information to know why your job isn't
> >> rechecking URLs.  Please tell me how your job is configured,
> specifically
> >> whether it's continuous or not.  Thanks.
> >>
> >> Karl
> >>
> >>
> >> On Mon, Sep 2, 2019 at 4:47 AM Priya Arora  wrote:
> >>
> >> > Hi,
> >> >
> >> > I have a query regarding manifoldCF. Is this having some kind of
> >> > functionality to check, if the URL it is crawling, does exist actually
> >> or
> >> > page not found(404).
> >> >
> >> > Like I have a requirement in which i am crawling data for university
> and
> >> > job i continuously running.After some period it found that the certain
> >> > URL's have been removed from University site but its is getting
> indexed
> >> > still also.
> >> >
> >> > Some pages have been marked as status 404.
> >> >  How can manifold be automatise to check this , that if the URL is
> >> > corresponding to 404(does not  exist anymore), it should be indexed
> >> >
> >> > Thanks
> >> > Priya.
> >> >
> >>
> >
>