Re: Login with Kerberos keytab ?

2016-02-04 Thread Niels Basjes
Hi Rohini, I created a jira : https://issues.apache.org/jira/browse/PIG-4796 Niels Basjes On Tue, Jan 19, 2016 at 7:19 AM, Rohini Palaniswamy <rohini.adi...@gmail.com > wrote: > Can't you set up a cron to kinit periodically? If you need pig to do it, > it will have to be a ne

Re: Login with Kerberos keytab ?

2016-01-24 Thread Niels Basjes
e clients (hadoop, pig, hive) do > it now. > > On Wed, Jan 6, 2016 at 6:35 AM, Niels Basjes <ni...@basjes.nl> wrote: > > > Hi, > > > > When I run a Pig job on a Kerberos secured cluster it uses the tickets > > obtained from the kinit I did just before starti

Login with Kerberos keytab ?

2016-01-06 Thread Niels Basjes
I checked the source of Pig and couldn't find any reference to logging in with a keytab file. How can I use a keytab file to authenticate a pig job on a secure cluster? Or is this for which I should submit a feature request? -- Best regards / Met vriendelijke groeten, Niels Basjes

Re: Using secure HBase from Pig UDF?

2015-06-08 Thread Niels Basjes
Thanks, we'll give this a try! Niels Basjes On Mon, Jun 8, 2015 at 10:04 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Niels, I plan to have PIG-3038 in next two weeks which should simplify accessing secure hbase, but it will only be in 0.16 and that is at least 3-4 months away

Using secure HBase from Pig UDF?

2015-05-28 Thread Niels Basjes
vriendelijke groeten, Niels Basjes

Terminating a pig flow on missing data

2015-03-19 Thread Niels Basjes
like (but haven't found yet) is a check in a Pig job where I can say something like Terminate the entire flow if this set is less than 1000 records. What is the best way to implement such a 'safety net'? -- Best regards / Met vriendelijke groeten, Niels Basjes

Pig -x tez fails over tez.session.local-resources.pb.file-name ??

2015-02-26 Thread Niels Basjes
have done some googleing on that term and I was unable to figure out what went wrong. Do you guys have any suggestions what I have done wrong? Thanks. Niels Basjes 2015-02-26 17:15:53,310 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: fs.default.name is deprecated. Instead

Re: Pig Register Command: Fetching from Repository

2015-02-12 Thread Niels Basjes
/ Met vriendelijke groeten, Niels Basjes

Re: Example of a Loader returning a typed map

2015-01-31 Thread Niels Basjes
do I fill the Tuple). In addition an example that shows this in combination with having a pushProjection that says it only wants a specific field in the record in the map would be really great. Has anyone done something like this before (in Java)? Niels Basjes On Fri, Jan 30, 2015 at 7:52 PM

Example of a Loader returning a typed map

2015-01-26 Thread Niels Basjes
. Thanks. -- Best regards Niels Basjes

Re: Bug when loading multiple files?

2014-12-27 Thread Niels Basjes
Do you get the same effect if you specify exactly one of those? If this then also happens and these files are these files gzipped: you may be using a buggy inputformat. See this https://issues.apache.org/jira/plugins/servlet/mobile#issue/MAPREDUCE-2094 On Dec 24, 2014 5:54 PM, Rodrigo Ferreira

Re: Testing using mock.Storage?

2014-11-25 Thread Niels Basjes
/groupId artifactIdpig/artifactId version0.13.0/version classifierh2/classifier /dependency Niels Basjes On Sun, Nov 23, 2014 at 8:29 PM, Niels Basjes ni...@basjes.nl wrote: I did some additional testing myself and I now have the smallest possible way to reproduce the effect I

Re: Testing using mock.Storage?

2014-11-23 Thread Niels Basjes
and java.lang.RuntimeException: No data for location 'output' at org.apache.pig.builtin.mock.Storage$Data.get(Storage.java:327) at org.apache.pig.builtin.mock.TestMockStorage.testMockStoreUnion(TestMockStorage.java:106) Any help suggestions to what I'm doing wrong is very welcome. Niels Basjes On Sun, Nov 23

Testing using mock.Storage?

2014-11-22 Thread Niels Basjes
/org/apache/pig/builtin/mock/TestMockStorage.java I couldn't get it to work so I copied the exact method testMockStoreAndLoad and I still get this error: *java.lang.RuntimeException: No data for location 'bar'* Does anyone have a suggestion on how to proceed? -- Best regards Niels Basjes

[Announce] Parsing Apache HTTPD logfiles made easy

2013-09-10 Thread Niels Basjes
of blind for the flaws. So I would really appreciate it if you guys can let me know what you think. Thanks. Niels Basjes Quote from the readme: *Usage (PIG) * You simply register the httpdlog-pigloader-1.0-SNAPSHOT-job.jar *REGISTER target

Re: Adding dependent jars for UDF in the PIG

2013-08-13 Thread Niels Basjes
I've had the exact same problem with my own udf and joda. It turn out that an old version of joda is embedded inside the pig application. My workaround was limitying myself to the old joda api. Niels On Aug 13, 2013 10:31 AM, Darpan R darpa...@gmail.com wrote: Thanks Serega and Ankit.

Re: ERROR 1071: Cannot convert a generic_writablecomparable to a String

2013-07-10 Thread Niels Basjes
AS (useragent:*chararray*, status:*chararray*); Niels On Wed, Jul 10, 2013 at 9:37 PM, Niels Basjes ni...@basjes.nl wrote: Hi, I'm writing a custom Loader for Pig and I'm stuck on the following problem. When I use this loader from a pig script and I do a DESCRIBE Clicks; I get *Clicks

Re: problems with .gz

2013-06-10 Thread Niels Basjes
stream. Does that sound familiar? will On Sat, Jun 8, 2013 at 8:00 AM, William Oberman ober...@civicscience.com wrote: They are all *.gz, I confirmed that first :-) On Saturday, June 8, 2013, Niels Basjes wrote: What are the exact filenames you used

Re: problems with .gz

2013-06-07 Thread Niels Basjes
What are the exact filenames you used? The decompression of input files is based on the filename extention. Niels On Jun 7, 2013 11:11 PM, William Oberman ober...@civicscience.com wrote: I'm using pig 0.11.2. I had been processing ASCII files of json with schema: (key:chararray, columns:bag

Re: Json array parsing issue with Pig JsonLoader

2013-05-23 Thread Niels Basjes
pull 308 branch? -- Best regards / Met vriendelijke groeten, Niels Basjes

Re: Building pig and piggybank from CDH 4.2.0 sources?

2013-05-08 Thread Niels Basjes
the cloudera sources and went for the plain pig sources instead. Regarding the other error, I can't reproduce it with Apache Pig. I am using Ant 1.8.2. Hmm, I should try this with 1.8.2 then. Perhaps there is a bug in ant 1.9.0 -- Best regards / Met vriendelijke groeten, Niels Basjes

Re: Json array parsing issue with Pig JsonLoader

2013-05-08 Thread Niels Basjes
You guys could try to simply build this branch https://github.com/kevinweil/elephant-bird/pull/308 On May 8, 2013 6:54 PM, Zhu Wayne zhuw.chic...@gmail.com wrote: Thanks guys for your advice. I am on CDH4.2 now. If Elephant-bird doesn't work with CDH4, UDF is my way out or I can patch piggybank

Building pig and piggybank from CDH 4.2.0 sources?

2013-04-24 Thread Niels Basjes
groeten, Niels Basjes

PIG UDFs for parsing UserAgents

2013-04-20 Thread Niels Basjes
on how to create a junit test for this. So I would really appreciate it if someone from the pig community could have a look at it before I submit a pull request. Thanks. -- Best regards / Met vriendelijke groeten, Niels Basjes

Re: PIG UDFs for parsing UserAgents

2013-04-20 Thread Niels Basjes
/TestBuiltin.java Also, you can mock up input/ouput data using http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/builtin/mock/Storage.html . On Sat, Apr 20, 2013 at 3:09 PM, Niels Basjes ni...@basjes.nl wrote: Hi, I created an addition to the ua-parser project ( https

Classpath issues with a custom loadfunc

2013-04-16 Thread Niels Basjes
regards / Met vriendelijke groeten, Niels Basjes