[Nutch Wiki] Update of "bin/nutch webapp" by kamaci
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Nutch Wiki" for change
notification.
The "bin/nutch webapp" page has been changed by kamaci:
https://wiki.apache.org/nutch/bin/nutch%20webapp?action=diff&rev1=3&rev2=4
= Nutch 2.X =
Invoking the webapp starts a NutchUiServer on port 8080. There is a
[[https://issues.apache.org/jira/browse/NUTCH-1859|Jira ticket available for
making the port configurable]]. The WebApp interacts directly with the Nutch
2.X REST API for configuring and administering your Nutch crawler. In it's
current form, this webapp is designed to provide Nutch administration and
configuration. The webapp is not designed to be an end-to-end system or a
search application (yet).
+
+ Webapp has an authentication support with
[[https://issues.apache.org/jira/browse/NUTCH-2344|Jira ticket for
Authentication Support for Web GUI]]. Default username and password
configurations are:
+
+ {{{
+ admin|admin
+ user|user
+ }}}
+
+ One can customise it at nutch-site.xml.
If you feel like contributing towards improving the WebApp, please checkout
the
[[https://issues.apache.org/jira/browse/NUTCH/component/11630/?selectedTab=com.atlassian.jira.jira-projects-plugin:component-summary-panel|webapp
Jira issues]].
[jira] [Updated] (NUTCH-2353) Create seed file with metadata using the REST API
[
https://issues.apache.org/jira/browse/NUTCH-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge Luis Betancourt Gonzalez updated NUTCH-2353:
--
Description:
At the moment its not possible to create a seed file and specify any metadata
when using the REST API. The file gets created but there is no option to add
any metadata to the seed URLs.
If we use a payload like this:
{code}
{
"name":"name-of-seedlist",
"seedUrls":[
{
"url" : "http://example.com";,
"metadata" : {
"key1" : "value1",
"key2" : "value2",
"key3" : "value3"
}
}
]
}
{code}
It should be easy to specify the desired metadata. Also this should keep BC
with the previous array syntax if we only want to specify the list of URLs
without any metadata at all.
was:
At the moment its not possible to create a seed file and specify any metadata
when using the REST API. The file gets created but there is no option to add
any metadata to the seed URLs.
If we use a payload like this:
{code}
{
"name":"name-of-seedlist",
"seedUrls":[
{
"url" : "http://example.com";,
"metadata" : {
"key1" : "value1",
"key2" : "value2",
"key3" : "value3"
}
}
]
}
{code}
It should be easy to specify the desired metadata.
> Create seed file with metadata using the REST API
> -
>
> Key: NUTCH-2353
> URL: https://issues.apache.org/jira/browse/NUTCH-2353
> Project: Nutch
> Issue Type: Improvement
> Components: injector, REST_api
>Affects Versions: 1.12
>Reporter: Jorge Luis Betancourt Gonzalez
>Assignee: Jorge Luis Betancourt Gonzalez
>Priority: Minor
> Labels: rest_api
> Fix For: 1.13
>
>
> At the moment its not possible to create a seed file and specify any metadata
> when using the REST API. The file gets created but there is no option to add
> any metadata to the seed URLs.
> If we use a payload like this:
> {code}
> {
> "name":"name-of-seedlist",
> "seedUrls":[
> {
> "url" : "http://example.com";,
> "metadata" : {
> "key1" : "value1",
> "key2" : "value2",
> "key3" : "value3"
> }
> }
> ]
> }
> {code}
> It should be easy to specify the desired metadata. Also this should keep BC
> with the previous array syntax if we only want to specify the list of URLs
> without any metadata at all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NUTCH-2353) Create seed file with metadata using the REST API
Jorge Luis Betancourt Gonzalez created NUTCH-2353:
-
Summary: Create seed file with metadata using the REST API
Key: NUTCH-2353
URL: https://issues.apache.org/jira/browse/NUTCH-2353
Project: Nutch
Issue Type: Improvement
Components: injector, REST_api
Affects Versions: 1.12
Reporter: Jorge Luis Betancourt Gonzalez
Assignee: Jorge Luis Betancourt Gonzalez
Priority: Minor
Fix For: 1.13
At the moment its not possible to create a seed file and specify any metadata
when using the REST API. The file gets created but there is no option to add
any metadata to the seed URLs.
If we use a payload like this:
{code}
{
"name":"name-of-seedlist",
"seedUrls":[
{
"url" : "http://example.com";,
"metadata" : {
"key1" : "value1",
"key2" : "value2",
"key3" : "value3"
}
}
]
}
{code}
It should be easy to specify the desired metadata.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NUTCH-2352) Log with Generic Class Name at Nutch 1.x
[
https://issues.apache.org/jira/browse/NUTCH-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828683#comment-15828683
]
ASF GitHub Bot commented on NUTCH-2352:
---
GitHub user kamaci opened a pull request:
https://github.com/apache/nutch/pull/172
NUTCH-2352 Logging with generic class name.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kamaci/nutch NUTCH-2352
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nutch/pull/172.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #172
commit ea921b71ea2e1bf480b7417443fa04829c31cf82
Author: kamaci
Date: 2017-01-18T21:00:06Z
NUTCH-2352 Logging with generic class name.
> Log with Generic Class Name at Nutch 1.x
>
>
> Key: NUTCH-2352
> URL: https://issues.apache.org/jira/browse/NUTCH-2352
> Project: Nutch
> Issue Type: Improvement
>Affects Versions: 1.12
>Reporter: Furkan KAMACI
>Assignee: Furkan KAMACI
>Priority: Minor
> Fix For: 1.13
>
>
> There are many mistakes when some reference code is copied and created a new
> class and a logger is used. We can log with a generic class name to avoid it
> as like:
> {code:java}
> private static final Logger LOG =
> LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
> {code}
> (cf. SOLR-8324)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nutch pull request #172: NUTCH-2352 Logging with generic class name.
GitHub user kamaci opened a pull request: https://github.com/apache/nutch/pull/172 NUTCH-2352 Logging with generic class name. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kamaci/nutch NUTCH-2352 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nutch/pull/172.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #172 commit ea921b71ea2e1bf480b7417443fa04829c31cf82 Author: kamaci Date: 2017-01-18T21:00:06Z NUTCH-2352 Logging with generic class name. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NUTCH-2352) Log with Generic Class Name at Nutch 1.x
Furkan KAMACI created NUTCH-2352:
Summary: Log with Generic Class Name at Nutch 1.x
Key: NUTCH-2352
URL: https://issues.apache.org/jira/browse/NUTCH-2352
Project: Nutch
Issue Type: Improvement
Affects Versions: 1.12
Reporter: Furkan KAMACI
Assignee: Furkan KAMACI
Priority: Minor
Fix For: 1.13
There are many mistakes when some reference code is copied and created a new
class and a logger is used. We can log with a generic class name to avoid it as
like:
{code:java}
private static final Logger LOG =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
{code}
(cf. SOLR-8324)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NUTCH-2344) Authentication Support for Web GUI
[ https://issues.apache.org/jira/browse/NUTCH-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828445#comment-15828445 ] Hudson commented on NUTCH-2344: --- SUCCESS: Integrated in Jenkins build Nutch-nutchgora #1577 (See [https://builds.apache.org/job/Nutch-nutchgora/1577/]) NUTCH-2344 Authentication support for Web GUI (kamaci: rev def067735c5a6dc46d867c4c89cb176a275b1967) * (add) src/java/org/apache/nutch/webui/pages/auth/SignInPage.html * (add) src/java/org/apache/nutch/webui/pages/auth/SignInPage.java * (edit) ivy/ivy.xml * (edit) src/java/org/apache/nutch/webui/pages/assets/nutch-style.css * (add) src/java/org/apache/nutch/webui/pages/auth/SignInSession.java * (add) src/java/org/apache/nutch/webui/pages/auth/AuthenticatedWebPage.java * (add) src/java/org/apache/nutch/webui/pages/auth/package-info.java * (edit) conf/nutch-default.xml * (edit) src/java/org/apache/nutch/webui/pages/AbstractBasePage.java * (edit) src/java/org/apache/nutch/webui/NutchUiApplication.properties * (add) src/java/org/apache/nutch/webui/pages/auth/User.java * (edit) src/java/org/apache/nutch/webui/NutchUiApplication.java * (edit) src/java/org/apache/nutch/webui/pages/LogOutPage.java * (add) src/java/org/apache/nutch/webui/pages/auth/AuthorizationStrategy.java > Authentication Support for Web GUI > -- > > Key: NUTCH-2344 > URL: https://issues.apache.org/jira/browse/NUTCH-2344 > Project: Nutch > Issue Type: New Feature > Components: web gui >Affects Versions: 2.3.1 >Reporter: Furkan KAMACI >Assignee: Furkan KAMACI > Fix For: 2.4 > > Attachments: Firefox_Screenshot_2017-01-13T19-10-49.499Z.png > > > We should implement an authentication support for Web GUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NUTCH-2350) Add Missing activeConfId Field to NutchStatus Object
[ https://issues.apache.org/jira/browse/NUTCH-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828446#comment-15828446 ] Hudson commented on NUTCH-2350: --- SUCCESS: Integrated in Jenkins build Nutch-nutchgora #1577 (See [https://builds.apache.org/job/Nutch-nutchgora/1577/]) NUTCH-2350 Added missing activeConfId field to NutchStatus. (kamaci: rev 6e074fc0b61f421cb7bc516e92dea33c3ce23fd5) * (edit) src/java/org/apache/nutch/webui/client/model/NutchStatus.java > Add Missing activeConfId Field to NutchStatus Object > > > Key: NUTCH-2350 > URL: https://issues.apache.org/jira/browse/NUTCH-2350 > Project: Nutch > Issue Type: Bug > Components: web gui >Affects Versions: 2.3.1 >Reporter: Furkan KAMACI >Assignee: Furkan KAMACI > Fix For: 2.4 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NUTCH-2346) Check Types at Object Equality
[ https://issues.apache.org/jira/browse/NUTCH-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828417#comment-15828417 ] Lewis John McGibbney commented on NUTCH-2346: - [~kamaci] can you please substantiate on what the issue is here? We need to have Jira as the canonical provenance tracking for the issues. Thanks. > Check Types at Object Equality > -- > > Key: NUTCH-2346 > URL: https://issues.apache.org/jira/browse/NUTCH-2346 > Project: Nutch > Issue Type: Bug > Components: generator, metadata >Affects Versions: 2.3.1 >Reporter: Furkan KAMACI >Assignee: Furkan KAMACI >Priority: Minor > Fix For: 2.4 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NUTCH-2344) Authentication Support for Web GUI
[ https://issues.apache.org/jira/browse/NUTCH-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lewis John McGibbney resolved NUTCH-2344. - Resolution: Fixed Nice work [~kamaci] thank you > Authentication Support for Web GUI > -- > > Key: NUTCH-2344 > URL: https://issues.apache.org/jira/browse/NUTCH-2344 > Project: Nutch > Issue Type: New Feature > Components: web gui >Affects Versions: 2.3.1 >Reporter: Furkan KAMACI >Assignee: Furkan KAMACI > Fix For: 2.4 > > Attachments: Firefox_Screenshot_2017-01-13T19-10-49.499Z.png > > > We should implement an authentication support for Web GUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NUTCH-2350) Add Missing activeConfId Field to NutchStatus Object
[ https://issues.apache.org/jira/browse/NUTCH-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lewis John McGibbney resolved NUTCH-2350. - Resolution: Fixed Nice work [~kamaci] thank you > Add Missing activeConfId Field to NutchStatus Object > > > Key: NUTCH-2350 > URL: https://issues.apache.org/jira/browse/NUTCH-2350 > Project: Nutch > Issue Type: Bug > Components: web gui >Affects Versions: 2.3.1 >Reporter: Furkan KAMACI >Assignee: Furkan KAMACI > Fix For: 2.4 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NUTCH-2350) Add Missing activeConfId Field to NutchStatus Object
[ https://issues.apache.org/jira/browse/NUTCH-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828411#comment-15828411 ] ASF GitHub Bot commented on NUTCH-2350: --- Github user asfgit closed the pull request at: https://github.com/apache/nutch/pull/170 > Add Missing activeConfId Field to NutchStatus Object > > > Key: NUTCH-2350 > URL: https://issues.apache.org/jira/browse/NUTCH-2350 > Project: Nutch > Issue Type: Bug > Components: web gui >Affects Versions: 2.3.1 >Reporter: Furkan KAMACI >Assignee: Furkan KAMACI > Fix For: 2.4 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NUTCH-2344) Authentication Support for Web GUI
[ https://issues.apache.org/jira/browse/NUTCH-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828410#comment-15828410 ] ASF GitHub Bot commented on NUTCH-2344: --- Github user asfgit closed the pull request at: https://github.com/apache/nutch/pull/163 > Authentication Support for Web GUI > -- > > Key: NUTCH-2344 > URL: https://issues.apache.org/jira/browse/NUTCH-2344 > Project: Nutch > Issue Type: New Feature > Components: web gui >Affects Versions: 2.3.1 >Reporter: Furkan KAMACI >Assignee: Furkan KAMACI > Fix For: 2.4 > > Attachments: Firefox_Screenshot_2017-01-13T19-10-49.499Z.png > > > We should implement an authentication support for Web GUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nutch pull request #170: NUTCH-2350 Added missing activeConfId field to Nutc...
Github user asfgit closed the pull request at: https://github.com/apache/nutch/pull/170 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nutch pull request #163: NUTCH-2344 Authentication support for Web GUI
Github user asfgit closed the pull request at: https://github.com/apache/nutch/pull/163 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
ApacheCon CFP closing soon (11 February)
Hello, fellow Apache enthusiast. Thanks for your participation, and interest in, the projects of the Apache Software Foundation. I wanted to remind you that the Call For Papers (CFP) for ApacheCon North America, and Apache: Big Data North America, closes in less than a month. If you've been putting it off because there was lots of time left, it's time to dig for that inspiration and get those talk proposals in. It's also time to discuss with your developer and user community whether there's a track of talks that you might want to propose, so that you have more complete coverage of your project than a talk or two. We're looking for talks directly, and indirectly, related to projects at the Apache Software Foundation. These can be anything from in-depth technical discussions of the projects you work with, to talks about community, documentation, legal issues, marketing, and so on. We're also very interested in talks about projects and services built on top of Apache projects, and case studies of how you use Apache projects to solve real-world problems. We are particularly interested in presentations from Apache projects either in the Incubator, or recently graduated. ApacheCon is where people come to find out what technology they'll be using this time next year. Important URLs are: To submit a talk for Apache: Big Data - http://events.linuxfoundation.org/events/apache-big-data-north-america/program/cfp To submit a talk for ApacheCon - http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp To register for Apache: Big Data - http://events.linuxfoundation.org/events/apache-big-data-north-america/attend/register- To register for ApacheCon - http://events.linuxfoundation.org/events/apachecon-north-america/attend/register- Early Bird registration rates end March 12th, but if you're a committer on an Apache project, you get the low committer rate, which is less than half of the early bird rate! For further updated about ApacheCon, follow us on Twitter, @ApacheCon, or drop by our IRC channel, #apachecon on the Freenode IRC network. Or contact me - rbo...@apache.org - with any questions or concerns. Thanks! Rich Bowen, VP Conferences, Apache Software Foundation -- (You've received this email because you're on a dev@ or users@ mailing list of an Apache Software Foundation project. For subscription and unsubscription information, consult the headers of this email message, as this varies from one list to another.)
[jira] [Commented] (NUTCH-2333) Indexer for RabbitMQ
[ https://issues.apache.org/jira/browse/NUTCH-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828234#comment-15828234 ] Sebastian Nagel commented on NUTCH-2333: +1 looks good, although I haven't tested it. Yes, there is some overlap between indexer-rabbit and publish-rabbitmq, mostly regarding configuration of and connection to RabbitMQ. Eventually, code could be shared in a lib-rabbitmq plugin, now or as a later improvement. To implement indexer-rabbit as a NutchPublisher seems difficult: the IndexWriter and NutchPublisher interfaces are different, esp. how objects are serialized (a specific object "NutchDocument" vs. an unknown but universally JSON serializable object). Of course, one could think of indexing as a event, but in reality it's likely that different consumers/queues are used for monitoring and indexing content. > Indexer for RabbitMQ > > > Key: NUTCH-2333 > URL: https://issues.apache.org/jira/browse/NUTCH-2333 > Project: Nutch > Issue Type: New Feature > Components: indexer >Affects Versions: 1.12 >Reporter: Roannel Fernández Hernández >Priority: Minor > Fix For: 1.13 > > > A plugin to send the documents to a RabbitMQ server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
