Great will write up the doc link here, finish pirk63 then start this.
On Sep 19, 2016 5:34 PM, "Suneel Marthi" wrote:
> +100
>
> On Mon, Sep 19, 2016 at 11:24 PM, Ellison Anne Williams <
> eawilli...@apache.org> wrote:
>
> > Yes, ES is just an inputformat (like HDFS, Kafka, etc) - we don't need
So my concern with elastic search isn't really that I want it as a
submodule it's more that I don't think it belongs in any of the jars.
Otherwise one can argue every inputformat belongs. It seems more prudent
to have those included via some abstraction devs can add on thier own
(cascading's tap c
+100
On Mon, Sep 19, 2016 at 11:24 PM, Ellison Anne Williams <
eawilli...@apache.org> wrote:
> Yes, ES is just an inputformat (like HDFS, Kafka, etc) - we don't need a
> separate submodule.
>
> Aside from pirk-core, it seems that we would want to break the responder
> implementations out into sub
Yes, ES is just an inputformat (like HDFS, Kafka, etc) - we don't need a
separate submodule.
Aside from pirk-core, it seems that we would want to break the responder
implementations out into submodules. This would leave us with something
along the lines of the following (at this point):
pirk-core
Here's an example from the Flink project for how they go about new features
or system breaking API changes, we could start a similar process. The Flink
guys call these FLIP (Flink Improvement Proposal) and Kafka community
similarly has something called KLIP.
We could start a PLIP (??? :-) )
https
Sounds good.
On Mon, Sep 19, 2016 at 4:22 PM, Darin Johnson
wrote:
> Alright, that was in the spirit of what I was thinking when I did this.
>
> Why don't we take Tim's suggested improvements to my PR (I'll do the
> necessary cleanup) and at the same time just remove the platform argument
> alto
Sure will do tonight.
On Sep 19, 2016 5:07 PM, "Suneel Marthi" wrote:
> A shared Google doc would be more convenient than a bunch of Jiras. Its
> easier to comment and add notes that way.
>
>
> On Mon, Sep 19, 2016 at 10:38 PM, Darin Johnson
> wrote:
>
> > Suneel, I'll try to put a couple jiras
A shared Google doc would be more convenient than a bunch of Jiras. Its
easier to comment and add notes that way.
On Mon, Sep 19, 2016 at 10:38 PM, Darin Johnson
wrote:
> Suneel, I'll try to put a couple jiras on it tonight with my thoughts.
> Based off my pirk-63 I was able to pull spark and s
Pirk-ES is more of a Source input (to be more technically precise) so it
doesn't warrant being a spearate module of its own.
But I am not sure about pirk-hadoop and pirk-storm aren't they both
different responder impls (??)
ElasticSearch should just be an input source like HDFS and doesn't need t
Github user wraydulany commented on the issue:
https://github.com/apache/incubator-pirk/pull/95
Please don't close this until we have some feedback from the community
indicating that the changes provide sufficient background to make clear my
(previously quite obscure, unless you knew
GitHub user wraydulany opened a pull request:
https://github.com/apache/incubator-pirk/pull/95
[WIP] [PIRK-69] Improve clarity of group theory portion of slides.
First attempt at adding clarifying slides. The new slides are pages 4-9
You can merge this pull request into a Git reposi
Suneel, I'll try to put a couple jiras on it tonight with my thoughts.
Based off my pirk-63 I was able to pull spark and storm out with no
issues. I was planning to pull them out, then tackling elastic search,
then hadoop as it's a little entrenched. This should keep most PRs to
manageable chunks
Alright, that was in the spirit of what I was thinking when I did this.
Why don't we take Tim's suggested improvements to my PR (I'll do the
necessary cleanup) and at the same time just remove the platform argument
altogether since backwards compatibility isn't upsetting anyone.
We'll still need
One explicit vote, one implicit vote for updating/clarifying the slides.
I've created PIRK-69 to improve slide clarity.
Unless this doesn't make sense (tell me) I'll mark PRs on this as WIPs
until I've got some agreement from the community that the slides are clear
enough.
On Mon, Sep 19, 2016 a
Hey Walter / Tim,
I just wanted to add I had some trouble similar to Tim's when trying to
understand the Wideskies paper. As a person without a background in group
theory/theoretical math trying to get my head around this stuff, it was
very difficult for me to even find with Google what the nota
Correction:
...bby the binomial theorem, (1+N)**N = 1 + N*N + other terms divisible...
I multiplied by N on the left when I ought to have exponentiated
Walter
On Mon, Sep 19, 2016 at 1:36 PM, Walter Ray-Dulany
wrote:
> Hi Tim,
>
> Apologies! It's disorienting at first, and most of all when on
Hi Tim,
Apologies! It's disorienting at first, and most of all when one actually
tries to sit down and do a real example. The version on the slides was not
written in one go, I assure you.
Let's go through, and see what's not working.
**
> I'm trying a very simple exampl
Refactor is definitely a first priority. Is there a design/proposal draft
that we could comment on about how to go about refactoring the code. I
have been trying to keep up with the emails but definitely would have
missed some.
On Mon, Sep 19, 2016 at 6:57 PM, Ellison Anne Williams <
eawilli..
Agree - let's leave the config/CLI the way it is for now and tackle that as
a subsequent design discussion and PR.
Also, I think that we should leave the ResponderDriver and the
ResponderProps alone for this PR and push to a subsequent PR (once we
decide if and how we would like to delegate each).
On 19/09/16 15:46, Darin Johnson wrote:
> Hey guys,
>
> Thanks for looking at the PR, I apologize if it offended anyone's eyes:).
>
> I'm glad it generated some discussion about the configuration. I didn't
> really like where things were heading with the config. However, didn't
> want to create
On 17/09/16 06:42, wraydulany wrote:
> [Pirk 67] - Add Slide Deck to the Website Documentation Explaining the
> Mathematics of Pirk
Argh! Walter you are driving me mad! :-)
Thank you for putting up the slides, and I really want to grok this, but
I'm struggling to work through even the simple
Hey guys,
Thanks for looking at the PR, I apologize if it offended anyone's eyes:).
I'm glad it generated some discussion about the configuration. I didn't
really like where things were heading with the config. However, didn't
want to create to much scope creep.
I think any hierarchical config
On 19/09/16 13:40, Ellison Anne Williams wrote:
> It seems that it's the same idea as the ResponderLauncher with the service
> component added to maintain something akin to the 'platform'. I would
> prefer that we just did away with the platform notion altogether and make
> the ResponderDriver 'dum
On 19/09/16 13:39, Suneel Marthi wrote:
> The way this PR is now is so similar to how bad IBM SystemML which is a
> hackwork of hurriedly put together and something I have often pointed out
> to others as a clear example of "how not to design crappy software". See
> this gist of an example code sn
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-pirk/pull/94
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user ellisonanne commented on the issue:
https://github.com/apache/incubator-pirk/pull/94
+1 will merge now
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wis
Ok - so, I'm officially shamed ;)
I'm not a big fan of java CLI either as it tends to be heavyweight and
inflexible - it was just super fast to put together as a first pass. Happy
to consider other more flexible options... :)
On Mon, Sep 19, 2016 at 8:40 AM, Ellison Anne Williams <
eawilli...@apa
It seems that it's the same idea as the ResponderLauncher with the service
component added to maintain something akin to the 'platform'. I would
prefer that we just did away with the platform notion altogether and make
the ResponderDriver 'dumb'. We get around needing a platform-aware service
by re
The way this PR is now is so similar to how bad IBM SystemML which is a
hackwork of hurriedly put together and something I have often pointed out
to others as a clear example of "how not to design crappy software". See
this gist of an example code snippet from IBM SystemML -
https://gist.github.co
How about an approach like this?
https://github.com/tellison/incubator-pirk/tree/pirk-63
The "on-ramp" is the driver [1], which calls upon the service to find a
plug-in [2] that claims to implement the required platform responder,
e.g. [3].
The list of plug-ins is given in the provider's JAR f
Github user DarinJ commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/93#discussion_r79377189
--- Diff:
src/main/java/org/apache/pirk/responder/wideskies/ResponderDriver.java ---
@@ -49,83 +41,111 @@
public class ResponderDriver
{
Github user DarinJ commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/93#discussion_r79377024
--- Diff:
src/main/java/org/apache/pirk/responder/wideskies/ResponderDriver.java ---
@@ -49,83 +41,111 @@
public class ResponderDriver
{
Github user ellisonanne commented on the issue:
https://github.com/apache/incubator-pirk/pull/93
A few other comments for discussion:
First, I am not opposed to having separate ResponderDrivers for each
responder, but let's think it through and see if we really need to go down
Github user ellisonanne commented on the issue:
https://github.com/apache/incubator-pirk/pull/93
+1 - looks good so far.
One item for consideration: I am in favor of *not* providing backwards
compatibility with the 'platform' option at this point, i.e. removing it
altogether
;-)
On 19/09/16 12:35, Walter Ray-Dulany wrote:
> I concede that in every way used in English (the only language i was
> considering), it's not a1 bit change; only if I'm allowed to abuse meaning
> to contort those words into '1 bit addition' could my sentence even
> remotely be considered correct
I have not seen/experienced similar issues, but I am fine with rolling it
back...
On Mon, Sep 19, 2016 at 6:05 AM, Tim Ellison wrote:
> I have intermittent failures caused by
> "PIRK-35 Execute Tests in Parallel"
>
> such as
>
>
> ---
> T E
I concede that in every way used in English (the only language i was
considering), it's not a1 bit change; only if I'm allowed to abuse meaning
to contort those words into '1 bit addition' could my sentence even
remotely be considered correct.
:( :)
On Sep 19, 2016 5:33 AM, "Tim Ellison" wrote:
I have intermittent failures caused by
"PIRK-35 Execute Tests in Parallel"
such as
---
T E S T S
---
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to crea
Github user smarthi commented on the issue:
https://github.com/apache/incubator-pirk/pull/94
+1 to merge
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
GitHub user tellison opened a pull request:
https://github.com/apache/incubator-pirk/pull/94
Update a number of Pirk's pom dependencies.
- move Pirk to later versions of JMH, Hadoop, commons-math3, commons-net,
json-simple, jacoco-maven-plugin, coveralls-maven-plugin, Surefire,
ma
Caution: extreme trivia below.. :-)
On 17/09/16 05:02, wraydulany wrote:
> GitHub user wraydulany opened a pull request:
>
> https://github.com/apache/incubator-pirk/pull/91
>
> [Pirk-66] - Fix Website To Indicate Pirk's Use of Java 8
>
> This is a one-byte change. Heck, it's a one-
Darin,
Unless I'm reading this wrong, the patch still has many references from
the ResponderDriver to the set of currently supported responders. This
code will have to change when somebody wants to add a new responder type.
I thought the plan was to have the responder driver agnostic of the
resp
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/93#discussion_r79352002
--- Diff:
src/main/java/org/apache/pirk/responder/wideskies/ResponderDriver.java ---
@@ -49,83 +41,111 @@
public class ResponderDriver
{
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/93#discussion_r79351660
--- Diff:
src/main/java/org/apache/pirk/responder/wideskies/ResponderDriver.java ---
@@ -49,83 +41,111 @@
public class ResponderDriver
{
44 matches
Mail list logo
