Re: kafka to hdfs through apache nifi

[tejaswi]

I have checked the logs and the output in logs

 

not able to move the data



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/kafka-to-hdfs-through-apache-nifi-tp12343p12366.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

[GitHub] nifi pull request #590: NIFI-2138 Making AccessPolicy have a single RequestA...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/590


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: kafka to hdfs through apache nifi

[Pierre Villard]

Depending of how you installed NiFi, the log file you are looking for
should be /logs/nifi-app.log.

2016-06-28 19:39 GMT+02:00 tejaswi :

> Ya I have started them but still the data is not moving.
> and where to check the errors in the logs exactly?
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/kafka-to-hdfs-through-apache-nifi-tp12343p12363.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

[GitHub] nifi pull request #589: NIFI-1952 Update to revision-locking for users and g...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/589


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #589: NIFI-1952 Update to revision-locking for users and groups

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/589
  
@jtstorck looks good. However, I noticed that the revision version was 
being necessarily checked. Those are already verified in the Tenant and 
AccessPolicy resource. Also, I updated the error message too.


https://github.com/mcgilman/nifi/commit/d8edda457cfcb140e9a03fb2f652b8c17f257653


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #590: NIFI-2138 Making AccessPolicy have a single RequestAction

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/590
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #589: NIFI-1952 Update to revision-locking for users and g...

[jtstorck]

GitHub user jtstorck opened a pull request:

https://github.com/apache/nifi/pull/589

NIFI-1952 Update to revision-locking for users and groups

Adding user and group summary objects (TenantEntity)
Fixed ComponentEntity JSON mapping issues when the id field is null

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jtstorck/nifi NIFI-1952

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/589.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #589


commit 2d927c8cdfcd5cc45e9ac3098c4a17d8d5f11ffa
Author: Jeff Storck 
Date:   2016-06-27T23:49:34Z

NIFI-1952 Update to revision-locking for users and groups
Adding user and group summary objects (TenantEntity)
Fixed ComponentEntity JSON mapping issues when the id field is null




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68799710
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68795836
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68795669
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68795163
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

Re: Github mails

[Joe Witt]

Sean,

Make sense and I believe it aligns with Lars original intent and suggestion.

I've captured these items here https://issues.apache.org/jira/browse/NIFI-2137

But in summary will carry out the following tasks pending any further
recommendations that might alter that.

1) Request INFRA to create issues@nifi.a.o
2) Request INFRA to send all Github notifications to issues@nifi.a.o
3) Request INFRA to send all JIRA notifications to issues@nifi.a.o
4) Document on webpage for mailing lists the purpose of each mailing
list (dev, commits, issues, users)
5) Document on webpage for mailing lists a link to
https://lists.apache.org/list.html?nifi.apache.org

Thanks
Joe

On Tue, Jun 28, 2016 at 8:17 AM, Sean Busbey  wrote:
> My usual concern with sending github notifications off of dev@ is that
> not as many folks watch lists that are defined as a dumping ground for
> automated messages (like a commits@ or notifications@).
>
> However, I usually think of github notifications like JIRA
> notifications. AFAICT we already send all of the JIRA traffic to
> commits@, so it makes sense for folks who want that level of detail on
> project implementation to have a single place to go.
>
> At the same time, it might be hard for a new person to realize all
> these things are going to commits@. Maybe we could add a brief
> description to our mailing list page[1] of when folks should look to
> the different lists?
>
> There's enough project activity now on jira/github/commits that I'd be
> in favor of splitting out things that aren't actually commits to a
> notifications@ (or issues@ or whatever).
>
> Also, now that lists.apache is live, I think we should link to it as
> the "user friendly" list UI rather than nabble[2]. Folks do need an
> @apache email address to post through lists.apache, but I don't know
> how much we value the "ease of posting" feature of nabble.
>
> [1]: http://nifi.apache.org/mailing_lists.html
> [2]: https://lists.apache.org/list.html?dev@nifi.apache.org
>
> On Mon, Jun 27, 2016 at 2:31 AM, Lars Francke  wrote:
>> Hi everyone,
>>
>> there's lots of mails from the Github integration in the dev@ mailing list.
>> It's getting pretty overwhelming. Yes I've created a filter but everyone
>> would need to do that. Other projects have opted to move these kind of
>> things to a "issues" mailing list.
>>
>> What do you think about that?
>>
>> Cheers,
>> Lars

[GitHub] nifi issue #588: NIFI-2127 Adding support for managing the user-group relati...

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/588
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68782625
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68780468
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68780983
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[JPercivall]

Github user JPercivall commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68777286
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

Re: Github mails

[Joe Witt]

Received and awesomely fast reply from INFRA.

All of our github notifications do go to dev@nifi.a.o today and there
are a few you can control but they must all be the same.  The things
we see going to commits are because some of them cause comments to get
entered into JIRA as well which in turn sends updates to
commits@nifi.a.o

"You can have them sent to wherever you like, it is configurable on
git-wip where those emails go to.
Notification types are typically:
 - new pull request
 - comment on PR
 - comment on specific code
 - PR closed
"

Based on Lars request and the +1 notes that followed I will assume
consensus later today or tomorrow and file the request to have all
github items go to commits@nifi.a.o.  If we'd like to further split
those to issues later we can pursue that.

Thanks
Joe

On Tue, Jun 28, 2016 at 7:25 AM, Joe Witt  wrote:
> Here is an INFRA request so we can gather more info.
>   https://issues.apache.org/jira/browse/INFRA-12191
>
> My intent would be to just have all github notfications go to
> comm...@nifi.apache.org rather than make another mailing list.
>
> We could also switch to having a pure issues mailing list which would
> be for all JIRA filings and Github auto notifications and then ideally
> commits would be just for code commits to SVN/Git but i'm not sure
> that distinction is valuable.  What does seem valuable is keeping dev
> clear for true developer driven discussion.
>
> If others have an opinion they'd like to share on this please do.
>
> Thanks
> Joe
>
> On Tue, Jun 28, 2016 at 2:18 AM, Lars Francke  wrote:
>> Thank you!
>>
>> There is at least pull request and issue stuff going to dev@ which would be
>> better suited in a different list I think.
>>
>> On Tue, Jun 28, 2016 at 1:59 AM, Joe Witt  wrote:
>>
>>> Some GitHub things go to commits@nifi
>>>
>>> Will reach out to infra to learn more about our options.
>>>
>>> I was initially concerned about losing GitHub comments from dev but even
>>> those seem inconsistent already.
>>>
>>> Will share what I learn so we can make a good decision.
>>> On Jun 27, 2016 4:55 PM, "Jeff"  wrote:
>>>
>>> > I agree, there's a lot of GitHub "noise" on the dev list.
>>> >
>>> > +1
>>> >
>>> > On Mon, Jun 27, 2016 at 5:58 PM Joe Witt  wrote:
>>> >
>>> > > Am on phone but the ones I see go to commits@nifi.a.o
>>> > >
>>> > > Which ones are you referring to?
>>> > >
>>> > > Thanks
>>> > > Joe
>>> > > On Jun 27, 2016 11:45 AM, "Andy LoPresto" 
>>> wrote:
>>> > >
>>> > > > +1.
>>> > > >
>>> > > > Andy LoPresto
>>> > > > alopre...@apache.org
>>> > > > *alopresto.apa...@gmail.com *
>>> > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>> > > >
>>> > > > On Jun 27, 2016, at 12:31 AM, Lars Francke 
>>> > > wrote:
>>> > > >
>>> > > > Hi everyone,
>>> > > >
>>> > > > there's lots of mails from the Github integration in the dev@
>>> mailing
>>> > > > list.
>>> > > > It's getting pretty overwhelming. Yes I've created a filter but
>>> > everyone
>>> > > > would need to do that. Other projects have opted to move these kind
>>> of
>>> > > > things to a "issues" mailing list.
>>> > > >
>>> > > > What do you think about that?
>>> > > >
>>> > > > Cheers,
>>> > > > Lars
>>> > > >
>>> > > >
>>> > > >
>>> > >
>>> >
>>>

Re: Github mails

[Joe Witt]

Here is an INFRA request so we can gather more info.
  https://issues.apache.org/jira/browse/INFRA-12191

My intent would be to just have all github notfications go to
comm...@nifi.apache.org rather than make another mailing list.

We could also switch to having a pure issues mailing list which would
be for all JIRA filings and Github auto notifications and then ideally
commits would be just for code commits to SVN/Git but i'm not sure
that distinction is valuable.  What does seem valuable is keeping dev
clear for true developer driven discussion.

If others have an opinion they'd like to share on this please do.

Thanks
Joe

On Tue, Jun 28, 2016 at 2:18 AM, Lars Francke  wrote:
> Thank you!
>
> There is at least pull request and issue stuff going to dev@ which would be
> better suited in a different list I think.
>
> On Tue, Jun 28, 2016 at 1:59 AM, Joe Witt  wrote:
>
>> Some GitHub things go to commits@nifi
>>
>> Will reach out to infra to learn more about our options.
>>
>> I was initially concerned about losing GitHub comments from dev but even
>> those seem inconsistent already.
>>
>> Will share what I learn so we can make a good decision.
>> On Jun 27, 2016 4:55 PM, "Jeff"  wrote:
>>
>> > I agree, there's a lot of GitHub "noise" on the dev list.
>> >
>> > +1
>> >
>> > On Mon, Jun 27, 2016 at 5:58 PM Joe Witt  wrote:
>> >
>> > > Am on phone but the ones I see go to commits@nifi.a.o
>> > >
>> > > Which ones are you referring to?
>> > >
>> > > Thanks
>> > > Joe
>> > > On Jun 27, 2016 11:45 AM, "Andy LoPresto" 
>> wrote:
>> > >
>> > > > +1.
>> > > >
>> > > > Andy LoPresto
>> > > > alopre...@apache.org
>> > > > *alopresto.apa...@gmail.com *
>> > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> > > >
>> > > > On Jun 27, 2016, at 12:31 AM, Lars Francke 
>> > > wrote:
>> > > >
>> > > > Hi everyone,
>> > > >
>> > > > there's lots of mails from the Github integration in the dev@
>> mailing
>> > > > list.
>> > > > It's getting pretty overwhelming. Yes I've created a filter but
>> > everyone
>> > > > would need to do that. Other projects have opted to move these kind
>> of
>> > > > things to a "issues" mailing list.
>> > > >
>> > > > What do you think about that?
>> > > >
>> > > > Cheers,
>> > > > Lars
>> > > >
>> > > >
>> > > >
>> > >
>> >
>>

[GitHub] nifi pull request #525: NIFI-1976 - Windows Event Log native processor

[brosander]

Github user brosander commented on a diff in the pull request:

https://github.com/apache/nifi/pull/525#discussion_r68759636
  
--- Diff: 
nifi-nar-bundles/nifi-windows-event-log-bundle/nifi-windows-event-log-processors/src/main/java/org/apache/nifi/processors/windows/event/log/ConsumeWindowsEventLog.java
 ---
@@ -0,0 +1,282 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.windows.event.log;
+
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.Kernel32Util;
+import com.sun.jna.platform.win32.WinNT;
+import org.apache.commons.io.Charsets;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.windows.event.log.jna.ErrorLookup;
+import 
org.apache.nifi.processors.windows.event.log.jna.EventSubscribeXmlRenderingCallback;
+import org.apache.nifi.processors.windows.event.log.jna.WEvtApi;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"ingest", "event", "windows"})
+@TriggerSerially
+@CapabilityDescription("Registers a Windows Event Log Subscribe Callback 
to receive FlowFiles from Events on Windows.  These can be filtered via channel 
and XPath.")
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Will set 
a MIME type value of application/xml.")
+})
+public class ConsumeWindowsEventLog extends 
AbstractSessionFactoryProcessor {
+public static final String DEFAULT_CHANNEL = "System";
+public static final String DEFAULT_XPATH = "*";
+public static final int DEFAULT_MAX_BUFFER = 1024 * 1024;
+public static final int DEFAULT_MAX_QUEUE_SIZE = 1024;
+
+public static final PropertyDescriptor CHANNEL = new 
PropertyDescriptor.Builder()
+.name("channel")
+.displayName("Channel")
+.required(true)
+.defaultValue(DEFAULT_CHANNEL)
+.description("The Windows Event Log Channel to listen to.")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor QUERY = new 
PropertyDescriptor.Builder()
+.name("query")
+.displayName("XPath Query")
+.required(true)
+.defaultValue(DEFAULT_XPATH)
+.description("XPath Query to filter events. (See 
https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx 
for examples.)")
+

Big Data Ingest

[Rajib Mandal]

Hi all,


We have files (size of each files  600GB- 700 GB ).  We want to load (bulk 
upload)these files into HDFS.

We are planning to choose Apache NiFi to ingest high volume of data.

Could you please advise if Apache NiFi is the right technology for  the high 
volume  of data  ingest?

If we use hadoop fs -put commands to load (bulk upload)these files into HDFS, 
do you foresee  any performance differnce between hadoop fs -put  and Apache 
NiFi?

Also we have couple of questions here?

  *   What  is the difference between NIFI and PUT - in terms of architecture, 
performance, scalability, high availability, fault tolerance ?
  *   Do we have any comparison metrics for the following items -supported 
features, unsupported features, limitations, free GA version and availability 
for commercial purpose etc?
  *   What Hadoop distributions are supported by NIFI?

Thanks & regards,
Rajib

[GitHub] nifi pull request #577: [NIFI-2027] Responsive Tables

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/577


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #577: [NIFI-2027] Responsive Tables

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/577
  
Thanks @scottyaslan! This has been merged into master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Github mails

[Lars Francke]

Thank you!

There is at least pull request and issue stuff going to dev@ which would be
better suited in a different list I think.

On Tue, Jun 28, 2016 at 1:59 AM, Joe Witt  wrote:

> Some GitHub things go to commits@nifi
>
> Will reach out to infra to learn more about our options.
>
> I was initially concerned about losing GitHub comments from dev but even
> those seem inconsistent already.
>
> Will share what I learn so we can make a good decision.
> On Jun 27, 2016 4:55 PM, "Jeff"  wrote:
>
> > I agree, there's a lot of GitHub "noise" on the dev list.
> >
> > +1
> >
> > On Mon, Jun 27, 2016 at 5:58 PM Joe Witt  wrote:
> >
> > > Am on phone but the ones I see go to commits@nifi.a.o
> > >
> > > Which ones are you referring to?
> > >
> > > Thanks
> > > Joe
> > > On Jun 27, 2016 11:45 AM, "Andy LoPresto" 
> wrote:
> > >
> > > > +1.
> > > >
> > > > Andy LoPresto
> > > > alopre...@apache.org
> > > > *alopresto.apa...@gmail.com *
> > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > > >
> > > > On Jun 27, 2016, at 12:31 AM, Lars Francke 
> > > wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > there's lots of mails from the Github integration in the dev@
> mailing
> > > > list.
> > > > It's getting pretty overwhelming. Yes I've created a filter but
> > everyone
> > > > would need to do that. Other projects have opted to move these kind
> of
> > > > things to a "issues" mailing list.
> > > >
> > > > What do you think about that?
> > > >
> > > > Cheers,
> > > > Lars
> > > >
> > > >
> > > >
> > >
> >
>