http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/complex.mbox ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/complex.mbox b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/complex.mbox new file mode 100644 index 0000000..2aa4828 --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/complex.mbox @@ -0,0 +1,291 @@ +From core-user-return-14700-apmail-hadoop-core-user-archive=hadoop.apache....@hadoop.apache.org Mon Jun 01 04:28:28 2009 +Return-Path: <core-user-return-14700-apmail-hadoop-core-user-archive=hadoop.apache....@hadoop.apache.org> +Delivered-To: [email protected] +Received: (qmail 19921 invoked from network); 1 Jun 2009 04:28:28 -0000 +Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) + by minotaur.apache.org with SMTP; 1 Jun 2009 04:28:28 -0000 +Received: (qmail 84995 invoked by uid 500); 1 Jun 2009 04:28:38 -0000 +Delivered-To: [email protected] +Received: (qmail 84895 invoked by uid 500); 1 Jun 2009 04:28:38 -0000 +Mailing-List: contact [email protected]; run by ezmlm +Precedence: bulk +List-Help: <mailto:[email protected]> +List-Unsubscribe: <mailto:[email protected]> +List-Post: <mailto:[email protected]> +List-Id: <core-user.hadoop.apache.org> +Reply-To: [email protected] +Delivered-To: mailing list [email protected] +Received: (qmail 84885 invoked by uid 99); 1 Jun 2009 04:28:38 -0000 +Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) + by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 04:28:38 +0000 +X-ASF-Spam-Status: No, hits=1.2 required=10.0 + tests=SPF_NEUTRAL +X-Spam-Check-By: apache.org +Received-SPF: neutral (athena.apache.org: local policy) +Received: from [69.147.107.21] (HELO mrout2-b.corp.re1.yahoo.com) (69.147.107.21) + by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 04:28:26 +0000 +Received: from SNV-EXPF01.ds.corp.yahoo.com (snv-expf01.ds.corp.yahoo.com [207.126.227.250]) + by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id n514QYA6099963 + for <[email protected]>; Sun, 31 May 2009 21:26:35 -0700 (PDT) +DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; + h=received:user-agent:date:subject:from:to:message-id: + thread-topic:thread-index:in-reply-to:mime-version:content-type: + content-transfer-encoding:x-originalarrivaltime; + b=YVtSNdgjeeSBS1yY3XDolul49i+HrgNG7QszMo9LzGnrwejjgsl5+iUM6EiQgEpV +Received: from SNV-EXVS08.ds.corp.yahoo.com ([207.126.227.9]) by SNV-EXPF01.ds.corp.yahoo.com with Microsoft SMTPSVC(6.0.3790.3959); + Sun, 31 May 2009 21:26:34 -0700 +Received: from 10.66.92.213 ([10.66.92.213]) by SNV-EXVS08.ds.corp.yahoo.com ([207.126.227.58]) with Microsoft Exchange Server HTTP-DAV ; + Mon, 1 Jun 2009 04:26:33 +0000 +User-Agent: Microsoft-Entourage/12.17.0.090302 +Date: Mon, 01 Jun 2009 09:56:31 +0530 +Subject: Re: question about when shuffle/sort start working +From: Jothi Padmanabhan <[email protected]> +To: <[email protected]> +Message-ID: <c649564f.1435f%[email protected]> +Thread-Topic: question about when shuffle/sort start working +Thread-Index: AcnicSNoBw19cMU8UEaXwAdZ1YYhuw== +In-Reply-To: <[email protected]> +Mime-version: 1.0 +Content-type: text/plain; + charset="US-ASCII" +Content-transfer-encoding: 7bit +X-OriginalArrivalTime: 01 Jun 2009 04:26:34.0501 (UTC) FILETIME=[257EAB50:01C9E271] +X-Virus-Checked: Checked by ClamAV on apache.org + +When a Mapper completes, MapCompletionEvents are generated. Reducers try to +fetch map outputs for a given map only on the receipt of such events. + +Jothi + + +On 5/30/09 10:00 AM, "Jianmin Woo" <[email protected]> wrote: + +> Hi, +> I am being confused by the protocol between mapper and reducer. When mapper +> emitting the (key,value) pair done, is there any signal the mapper send out to +> hadoop framework in protocol to indicate that map is done and the shuffle/sort +> can begin for reducer? If there is no this signal in protocol, when the +> framework begin the shuffle/sort? +> +> Thanks, +> Jianmin +> +> +> +> + + +From core-user-return-14701-apmail-hadoop-core-user-archive=hadoop.apache....@hadoop.apache.org Mon Jun 01 05:31:14 2009 +Return-Path: <core-user-return-14701-apmail-hadoop-core-user-archive=hadoop.apache....@hadoop.apache.org> +Delivered-To: [email protected] +Received: (qmail 38243 invoked from network); 1 Jun 2009 05:31:14 -0000 +Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) + by minotaur.apache.org with SMTP; 1 Jun 2009 05:31:14 -0000 +Received: (qmail 15621 invoked by uid 500); 1 Jun 2009 05:31:24 -0000 +Delivered-To: [email protected] +Received: (qmail 15557 invoked by uid 500); 1 Jun 2009 05:31:24 -0000 +Mailing-List: contact [email protected]; run by ezmlm +Precedence: bulk +List-Help: <mailto:[email protected]> +List-Unsubscribe: <mailto:[email protected]> +List-Post: <mailto:[email protected]> +List-Id: <core-user.hadoop.apache.org> +Reply-To: [email protected] +Delivered-To: mailing list [email protected] +Received: (qmail 15547 invoked by uid 99); 1 Jun 2009 05:31:24 -0000 +Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) + by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 05:31:24 +0000 +X-ASF-Spam-Status: No, hits=2.2 required=10.0 + tests=HTML_MESSAGE,SPF_PASS +X-Spam-Check-By: apache.org +Received-SPF: pass (nike.apache.org: local policy) +Received: from [68.142.237.94] (HELO n9.bullet.re3.yahoo.com) (68.142.237.94) + by apache.org (qpsmtpd/0.29) with SMTP; Mon, 01 Jun 2009 05:31:11 +0000 +Received: from [68.142.237.88] by n9.bullet.re3.yahoo.com with NNFMP; 01 Jun 2009 05:30:50 -0000 +Received: from [67.195.9.82] by t4.bullet.re3.yahoo.com with NNFMP; 01 Jun 2009 05:30:49 -0000 +Received: from [67.195.9.99] by t2.bullet.mail.gq1.yahoo.com with NNFMP; 01 Jun 2009 05:30:49 -0000 +Received: from [127.0.0.1] by omp103.mail.gq1.yahoo.com with NNFMP; 01 Jun 2009 05:28:01 -0000 +X-Yahoo-Newman-Property: ymail-3 +X-Yahoo-Newman-Id: [email protected] +Received: (qmail 35264 invoked by uid 60001); 1 Jun 2009 05:30:49 -0000 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1243834249; bh=R8qzdi/IbLyO8UwpnaujDpT9E+6bJ7nkmZN2803EmRk=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=vq4c6RIDbkuLPYd8mirusIXf6DqTb/IeT55In7W00Y5Sxx1ZiXBb78yE9+TDfXJ0elsEZvqv4ocyvolGE0eGtyYeJA0mZikpRNu6pidxPNpCplOcLHBRz7YQ7iERwv3TagRlWy2Xd3oD9ZeV0A05P7WUOiNNX1PUUJD1IVdrEZo= +DomainKey-Signature:a=rsa-sha1; q=dns; c=nofws; + s=s1024; d=yahoo.com; + h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; + b=6HXZV98ON5vBwmE/xS8stVD0D2F4dkMY7a0suX5KVTb736JdR8G59mqBq/dWcpbFTLiCLtxi18LMb/dU1RKRGOEdn3l3j/jKXhBrhIgfg3qtNskPedXDKBvn7JGXiSkqpA/tUtPjvc0Uuk8/LaA01SQTz40Engg7nD8/EJdIAhA=; +Message-ID: <[email protected]> +X-YMail-OSG: KzhhrJYVM1m.MCS6vRpRP2ZZO2PrfnbngosELDCIa91ZqvhJph4RdmzfUW0jw9W04RCSch1K730bPohwNpNBIk2QR_zt4_mfbhfq7YEPkSoz9LSXG90P9vIo5Fc8qyZN0U6vA9gtdyGQTpN5ahvillUH9nAF0TMWv2SvZJLjPlQ0Z0p8oK8ltBwGTgLrM8Jtdn9D29yoRyi3_EpVOfdD9OP.EK50Vr1XwSUYMbnpZ0WGHMwd.Yig7A6Elwadm3YVbfOdx2mfrG.jQsUAxQjRBNvbrOM57.FaE11kHTe9aoBWSeihNg-- +Received: from [216.145.54.7] by web111010.mail.gq1.yahoo.com via HTTP; Sun, 31 May 2009 22:30:49 PDT +X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.10 +References: <c649564f.1435f%[email protected]> +Date: Sun, 31 May 2009 22:30:49 -0700 (PDT) +From: Jianmin Woo <[email protected]> +Subject: Re: question about when shuffle/sort start working +To: [email protected] +In-Reply-To: <c649564f.1435f%[email protected]> +MIME-Version: 1.0 +Content-Type: multipart/alternative; boundary="0-1193839393-1243834249=:35091" +X-Virus-Checked: Checked by ClamAV on apache.org + +--0-1193839393-1243834249=:35091 +Content-Type: text/plain; charset=us-ascii + +Thanks a lot for your explanation, Jothi. + +So is this event generated by hadoop framework? Is there any API in mapper to fire this event? Actually, I am thinking to implement a mapper that will emit some <key, value> pairs, then fire this event to let the reducer works, the same mapper task then emit some other <key, value> pairs and repeat. Do you think is this logic feasible by current API? + +Thanks, +Jianmin + + + + + +________________________________ +From: Jothi Padmanabhan <[email protected]> +To: [email protected] +Sent: Monday, June 1, 2009 12:26:31 PM +Subject: Re: question about when shuffle/sort start working + +When a Mapper completes, MapCompletionEvents are generated. Reducers try to +fetch map outputs for a given map only on the receipt of such events. + +Jothi + + +On 5/30/09 10:00 AM, "Jianmin Woo" <[email protected]> wrote: + +> Hi, +> I am being confused by the protocol between mapper and reducer. When mapper +> emitting the (key,value) pair done, is there any signal the mapper send out to +> hadoop framework in protocol to indicate that map is done and the shuffle/sort +> can begin for reducer? If there is no this signal in protocol, when the +> framework begin the shuffle/sort? +> +> Thanks, +> Jianmin +> +> +> +> + + + +--0-1193839393-1243834249=:35091-- + + +From core-user-return-14702-apmail-hadoop-core-user-archive=hadoop.apache....@hadoop.apache.org Mon Jun 01 06:04:30 2009 +Return-Path: <core-user-return-14702-apmail-hadoop-core-user-archive=hadoop.apache....@hadoop.apache.org> +Delivered-To: [email protected] +Received: (qmail 53387 invoked from network); 1 Jun 2009 06:04:29 -0000 +Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) + by minotaur.apache.org with SMTP; 1 Jun 2009 06:04:29 -0000 +Received: (qmail 39066 invoked by uid 500); 1 Jun 2009 06:04:39 -0000 +Delivered-To: [email protected] +Received: (qmail 38970 invoked by uid 500); 1 Jun 2009 06:04:39 -0000 +Mailing-List: contact [email protected]; run by ezmlm +Precedence: bulk +List-Help: <mailto:[email protected]> +List-Unsubscribe: <mailto:[email protected]> +List-Post: <mailto:[email protected]> +List-Id: <core-user.hadoop.apache.org> +Reply-To: [email protected] +Delivered-To: mailing list [email protected] +Received: (qmail 38955 invoked by uid 99); 1 Jun 2009 06:04:39 -0000 +Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) + by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 06:04:39 +0000 +X-ASF-Spam-Status: No, hits=1.2 required=10.0 + tests=SPF_NEUTRAL +X-Spam-Check-By: apache.org +Received-SPF: neutral (athena.apache.org: local policy) +Received: from [216.145.54.172] (HELO mrout2.yahoo.com) (216.145.54.172) + by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 06:04:28 +0000 +Received: from SNV-EXBH01.ds.corp.yahoo.com (snv-exbh01.ds.corp.yahoo.com [207.126.227.249]) + by mrout2.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id n5163FGq038852 + for <[email protected]>; Sun, 31 May 2009 23:03:15 -0700 (PDT) +DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; + h=received:user-agent:date:subject:from:to:message-id: + thread-topic:thread-index:in-reply-to:mime-version:content-type: + content-transfer-encoding:x-originalarrivaltime; + b=rChE4SCnwtWaZpjhovkiXDKfDiVNdRRvsadSGG9S9bgvOexn/9/5JjEQx1pOR7Nb +Received: from SNV-EXVS08.ds.corp.yahoo.com ([207.126.227.9]) by SNV-EXBH01.ds.corp.yahoo.com with Microsoft SMTPSVC(6.0.3790.3959); + Sun, 31 May 2009 23:03:15 -0700 +Received: from 10.66.92.213 ([10.66.92.213]) by SNV-EXVS08.ds.corp.yahoo.com ([207.126.227.58]) with Microsoft Exchange Server HTTP-DAV ; + Mon, 1 Jun 2009 06:03:15 +0000 +User-Agent: Microsoft-Entourage/12.17.0.090302 +Date: Mon, 01 Jun 2009 11:33:13 +0530 +Subject: Re: question about when shuffle/sort start working +From: Jothi Padmanabhan <[email protected]> +To: <[email protected]> +Message-ID: <c6496cf9.1437c%[email protected]> +Thread-Topic: question about when shuffle/sort start working +Thread-Index: AcnifqWrLG6N7GAk7kqy9QalVWfegQ== +In-Reply-To: <[email protected]> +Mime-version: 1.0 +Content-type: text/plain; + charset="US-ASCII" +Content-transfer-encoding: 7bit +X-OriginalArrivalTime: 01 Jun 2009 06:03:15.0462 (UTC) FILETIME=[A7231260:01C9E27E] +X-Virus-Checked: Checked by ClamAV on apache.org + + +No you cannot raise this event yourself, this event is generated internally +by the framework. + +I am guessing that what you probably want is to have a chain of MapReduce +Jobs where the output of one is automatically fed as input to another. You +can look at these classes: JobControl and ChainMapper/ChainReducer. + +Jothi + +On 6/1/09 11:00 AM, "Jianmin Woo" <[email protected]> wrote: + +> Thanks a lot for your explanation, Jothi. +> +> So is this event generated by hadoop framework? Is there any API in mapper to +> fire this event? Actually, I am thinking to implement a mapper that will emit +> some <key, value> pairs, then fire this event to let the reducer works, the +> same mapper task then emit some other <key, value> pairs and repeat. Do you +> think is this logic feasible by current API? +> +> Thanks, +> Jianmin +> +> +> +> +> +> ________________________________ +> From: Jothi Padmanabhan <[email protected]> +> To: [email protected] +> Sent: Monday, June 1, 2009 12:26:31 PM +> Subject: Re: question about when shuffle/sort start working +> +> When a Mapper completes, MapCompletionEvents are generated. Reducers try to +> fetch map outputs for a given map only on the receipt of such events. +> +> Jothi +> +> +> On 5/30/09 10:00 AM, "Jianmin Woo" <[email protected]> wrote: +> +>> Hi, +>> I am being confused by the protocol between mapper and reducer. When mapper +>> emitting the (key,value) pair done, is there any signal the mapper send out +>> to +>> hadoop framework in protocol to indicate that map is done and the +>> shuffle/sort +>> can begin for reducer? If there is no this signal in protocol, when the +>> framework begin the shuffle/sort? +>> +>> Thanks, +>> Jianmin +>> +>> +>> +>> +> +> +> + +
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-sessions.log ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-sessions.log b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-sessions.log new file mode 100644 index 0000000..633a15c --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-sessions.log @@ -0,0 +1,9 @@ +Started GET /foo + Foo Started GET as HTML +Completed 401 Unauthorized in 0ms + + +Started GET /bar + Bar as HTML +Completed 200 OK in 339ms +Started GET /baz http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace-expected-long-event.log ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace-expected-long-event.log b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace-expected-long-event.log new file mode 100644 index 0000000..419f799 --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace-expected-long-event.log @@ -0,0 +1,25 @@ +Grave: Timer task com.base2services.jenkins.SqsQueueHandler@32eea79d failed +com.amazonaws.AmazonClientException: Unable to calculate a request signature: Unable to calculate a request signature: Empty key + at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:71) + at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:55) + at com.amazonaws.auth.QueryStringSigner.sign(QueryStringSigner.java:83) + at com.amazonaws.auth.QueryStringSigner.sign(QueryStringSigner.java:46) + at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:238) + at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:170) + at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:776) + at com.amazonaws.services.sqs.AmazonSQSClient.listQueues(AmazonSQSClient.java:564) + at com.amazonaws.services.sqs.AmazonSQSClient.listQueues(AmazonSQSClient.java:732) + at com.base2services.jenkins.SqsProfile.createQueue(SqsProfile.java:72) + at com.base2services.jenkins.SqsProfile.getQueueUrl(SqsProfile.java:62) + at com.base2services.jenkins.SqsQueueHandler.doRun(SqsQueueHandler.java:37) + at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) + at java.util.TimerThread.mainLoop(Timer.java:555) + at java.util.TimerThread.run(Timer.java:505) +Caused by: com.amazonaws.AmazonClientException: Unable to calculate a request signature: Empty key + at com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:90) + at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:68) + ... 14 more +Caused by: java.lang.IllegalArgumentException: Empty key + at javax.crypto.spec.SecretKeySpec.<init>(SecretKeySpec.java:96) + at com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:87) + ... 15 more \ No newline at end of file http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace.log ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace.log b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace.log new file mode 100644 index 0000000..b860012 --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace.log @@ -0,0 +1,30 @@ +juil. 25, 2012 10:49:46 AM hudson.triggers.SafeTimerTask run +Grave: Timer task com.base2services.jenkins.SqsQueueHandler@32eea79d failed +com.amazonaws.AmazonClientException: Unable to calculate a request signature: Unable to calculate a request signature: Empty key + at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:71) + at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:55) + at com.amazonaws.auth.QueryStringSigner.sign(QueryStringSigner.java:83) + at com.amazonaws.auth.QueryStringSigner.sign(QueryStringSigner.java:46) + at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:238) + at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:170) + at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:776) + at com.amazonaws.services.sqs.AmazonSQSClient.listQueues(AmazonSQSClient.java:564) + at com.amazonaws.services.sqs.AmazonSQSClient.listQueues(AmazonSQSClient.java:732) + at com.base2services.jenkins.SqsProfile.createQueue(SqsProfile.java:72) + at com.base2services.jenkins.SqsProfile.getQueueUrl(SqsProfile.java:62) + at com.base2services.jenkins.SqsQueueHandler.doRun(SqsQueueHandler.java:37) + at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) + at java.util.TimerThread.mainLoop(Timer.java:555) + at java.util.TimerThread.run(Timer.java:505) +Caused by: com.amazonaws.AmazonClientException: Unable to calculate a request signature: Empty key + at com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:90) + at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:68) + ... 14 more +Caused by: java.lang.IllegalArgumentException: Empty key + at javax.crypto.spec.SecretKeySpec.<init>(SecretKeySpec.java:96) + at com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:87) + ... 15 more + + +juil. 25, 2012 10:49:54 AM hudson.slaves.SlaveComputer tryReconnect +Infos: Attempting to reconnect CentosVagrant \ No newline at end of file http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/non-length-delimited-20130430-234145-tweets.json.gz ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/non-length-delimited-20130430-234145-tweets.json.gz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/non-length-delimited-20130430-234145-tweets.json.gz new file mode 100644 index 0000000..e5c0f7a Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/non-length-delimited-20130430-234145-tweets.json.gz differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/rsstest.rss ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/rsstest.rss b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/rsstest.rss new file mode 100644 index 0000000..758f6a1 --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/rsstest.rss @@ -0,0 +1,36 @@ +<?xml version="1.0" encoding="ISO-8859-1" ?> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> +<rss version="0.91"> + <channel> + <title>TestChannel</title> + <link>http://test.channel.com/</link> + <description>Sample RSS File for Junit test</description> + <language>en-us</language> + + <item> + <title>Home Page of Chris Mattmann</title> + <link>http://www-scf.usc.edu/~mattmann/</link> + <description>Chris Mattmann's home page</description> + </item> + <item> + <title>Awesome Open Source Search Engine</title> + <link>http://www.nutch.org/</link> + <description>Yup, that's what it is</description> + </item> + </channel> +</rss> http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433 ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433 b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433 new file mode 100644 index 0000000..380ab6e --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433 @@ -0,0 +1,4 @@ +1000 +{"text":"sample tweet one","retweet_count":0,"in_reply_to_user_id":null,"retweeted":false,"truncated":false,"source":"href=\"http:\/\/sample.com\"","id_str":"1234567891","entities":{"user_mentions":[],"hashtags":[],"urls":[]},"in_reply_to_status_id":null,"place":null,"in_reply_to_status_id_str":null,"coordinates":null,"created_at":"Wed Sep 05 01:01:01 +0000 1985","in_reply_to_screen_name":null,"favorited":false,"in_reply_to_user_id_str":null,"user":{"default_profile_image":false,"friends_count":111,"profile_background_color":"3C0C29","location":"Palo Alto","is_translator":false,"profile_background_tile":true,"favourites_count":11,"verified":false,"profile_sidebar_fill_color":"efefef","follow_request_sent":null,"contributors_enabled":false,"description":"desc1","profile_sidebar_border_color":"eeeeee","profile_image_url_https":"https:\/\/si0.twimg.com\/profile_images\/1\/normal.jpg","id_str":"1111111","listed_count":1,"lang":"en","screen_name":"fake_user1","show_all_inline_media":fals e,"profile_use_background_image":true,"profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/1111111\/normal.jpg","default_profile":false,"statuses_count":11111,"created_at":"Thu Apr 07 11:04:54 +0000 1985","profile_text_color":"333333","followers_count":111,"protected":false,"following":null,"notifications":null,"profile_background_image_url":"http:\/\/a0.twimg.com\/images\/themes\/theme1\/bg.gif","time_zone":null,"url":null,"name":"name1","geo_enabled":false,"profile_link_color":"009999","id":1111112,"profile_background_image_url_https":"https:\/\/si0.twimg.com\/images\/themes\/theme1\/bg.gif","utc_offset":null},"id":11111112,"contributors":null,"geo":null} +2000 +{"text":"sample tweet two","retweet_count":0,"in_reply_to_user_id":null,"retweeted":false,"truncated":false,"source":"href=\"http:\/\/sample.com\"","id_str":"2345678902","entities":{"user_mentions":[],"hashtags":[],"urls":[]},"in_reply_to_status_id":null,"place":null,"in_reply_to_status_id_str":null,"coordinates":null,"created_at":"Wed Sep 05 02:14:34 +0000 1985","in_reply_to_screen_name":null,"favorited":false,"in_reply_to_user_id_str":null,"user":{"default_profile_image":false,"friends_count":222,"profile_background_color":"3C0C29","location":"San Francisco","is_translator":false,"profile_background_tile":false,"favourites_count":22,"verified":false,"profile_sidebar_fill_color":"B2D948","follow_request_sent":null,"contributors_enabled":false,"description":"desc2","profile_sidebar_border_color":"8EC63D","profile_image_url_https":"https:\/\/si0.twimg.com\/profile_images\/22222222\/image_normal.jpg","id_str":"2222222","listed_count":0,"lang":"en","screen_name":"fake_user2","show_all_ inline_media":false,"profile_use_background_image":true,"profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/2222222\/image_normal.jpg","default_profile":false,"statuses_count":222222,"created_at":"Thu Aug 04 11:33:28 +0000 1985","profile_text_color":"444444","followers_count":222,"protected":false,"following":null,"notifications":null,"profile_background_image_url":"http:\/\/a0.twimg.com\/profile_background_images\/222222\/222222.jpg","time_zone":"Central Time (US & Canada)","url":null,"name":"name2","geo_enabled":false,"profile_link_color":"9A0057","id":2222223,"profile_background_image_url_https":"https:\/\/si0.twimg.com\/profile_background_images\/2222222\/22222.jpg","utc_offset":-21600},"id":222223,"contributors":null,"geo":null} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-medium.avro ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-medium.avro b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-medium.avro new file mode 100644 index 0000000..900507c Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-medium.avro differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-subschema.avsc ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-subschema.avsc b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-subschema.avsc new file mode 100644 index 0000000..cd64717 --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-subschema.avsc @@ -0,0 +1,12 @@ +{ + "type" : "record", + "name" : "Doc", + "doc" : "adoc", + "fields" : [ { + "name" : "id", + "type" : "string" + }, { + "name" : "text", + "type" : [ "string", "null" ] + } ] +} http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avro ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avro b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avro new file mode 100644 index 0000000..4dbf180 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avro differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avsc ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avsc b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avsc new file mode 100644 index 0000000..9e4529f --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avsc @@ -0,0 +1,57 @@ +{ + "type" : "record", + "name" : "Doc", + "doc" : "adoc", + "fields" : [ { + "name" : "id", + "type" : "string" + }, { + "name" : "user_friends_count", + "type" : [ "int", "null" ] + }, { + "name" : "user_location", + "type" : [ "string", "null" ] + }, { + "name" : "user_description", + "type" : [ "string", "null" ] + }, { + "name" : "user_statuses_count", + "type" : [ "int", "null" ] + }, { + "name" : "user_followers_count", + "type" : [ "int", "null" ] + }, { + "name" : "user_name", + "type" : [ "string", "null" ] + }, { + "name" : "user_screen_name", + "type" : [ "string", "null" ] + }, { + "name" : "created_at", + "type" : [ "string", "null" ] + }, { + "name" : "text", + "type" : [ "string", "null" ] + }, { + "name" : "retweet_count", + "type" : [ "int", "null" ] + }, { + "name" : "retweeted", + "type" : [ "boolean", "null" ] + }, { + "name" : "in_reply_to_user_id", + "type" : [ "long", "null" ] + }, { + "name" : "source", + "type" : [ "string", "null" ] + }, { + "name" : "in_reply_to_status_id", + "type" : [ "long", "null" ] + }, { + "name" : "media_url_https", + "type" : [ "string", "null" ] + }, { + "name" : "expanded_url", + "type" : [ "string", "null" ] + } ] +} http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.bz2 ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.bz2 b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.bz2 new file mode 100644 index 0000000..a4a9159 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.bz2 differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.gz ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.gz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.gz new file mode 100644 index 0000000..3e7a44c Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.gz differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.7z ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.7z b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.7z new file mode 100644 index 0000000..94d62d3 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.7z differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.cpio ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.cpio b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.cpio new file mode 100644 index 0000000..c13a1cb Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.cpio differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tar ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tar b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tar new file mode 100644 index 0000000..3076a58 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tar differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tbz2 ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tbz2 b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tbz2 new file mode 100644 index 0000000..21488d3 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tbz2 differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tgz ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tgz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tgz new file mode 100644 index 0000000..baca6bb Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tgz differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.zip ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.zip b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.zip new file mode 100644 index 0000000..27d600d Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.zip differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-outlook.msg ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-outlook.msg b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-outlook.msg new file mode 100644 index 0000000..c975c0c Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-outlook.msg differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-zip-of-zip.zip ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-zip-of-zip.zip b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-zip-of-zip.zip new file mode 100644 index 0000000..f6b3edc Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-zip-of-zip.zip differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testAIFF.aif ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testAIFF.aif b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testAIFF.aif new file mode 100644 index 0000000..97eac1d Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testAIFF.aif differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMP.bmp ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMP.bmp b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMP.bmp new file mode 100644 index 0000000..c017615 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMP.bmp differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt new file mode 100644 index 0000000..b0f1a68 --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt @@ -0,0 +1,3 @@ +BMW to Make Hybrid Sports Carâ© + By CHRISTOPH RAUWALD . +LEIPZIG, GermanyâGerman car maker BMW AG said Friday it will start series production of a new plug-in hybrid sports car in 2013, to be based on the Vision EfficientDynamics Concept car shown at the Frankfurt auto show in September last year. Chief Executive Norbert Reithofer said the car will be produced in Germany but didn't provide details on the price. The BMW Vision EfficientDynamics Concept car is a sporty plug-in, full hybrid with a turbo-diesel engine, four seats and upward-pivoting doors. BMW executive board member Klaus Draeger told reporters he expects to achieve "a significant sales volume" with the new high-performance sports car. Asked whether annual sales could exceed 1,000 vehicles, Mr. Draeger said, "You said this and I'm not saying this is wrong." In March, Mr. Reithofer indicated that the concept car was set to make it into series production. "I like the car. And you know what it means when I say I like the carâit means I will drive it. It's not just a concept car," he told analysts during a presentation in Munich. The car will be designed for sale in all major global markets, which according to Mr. Draeger might require offering a gasoline engine instead of the prototype's three-cylinder diesel engine. Diesel cars account for roughly half of the European market, but are significantly less popular in the U.S. and hardly present at all in China. Mr. Draeger declined to comment on the vehicle's price tag, but noted that in order to achieve substantial sales volumes the price mustn't be too high. He said the same goes for BMW's planned Megacity Vehicle. A price tag of â¬60,000 ($85,242) or more would certainly limit potential sales volumes, he said. http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testDITA.dita ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testDITA.dita b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testDITA.dita new file mode 100644 index 0000000..b68da7b --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testDITA.dita @@ -0,0 +1,34 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE task PUBLIC "-//OASIS//DTD DITA Task//EN" "task.dtd"> +<task id="apache-tika"> + <title>Apache Tika</title> + <shortdesc>Apache Tika - a content analysis toolkit.</shortdesc> + <prolog> + <author>Apache Software Foundation</author> + <copyright> + <copyryear year="2011"/> + <copyrholder>Apache Software Foundation</copyrholder> + </copyright> + <metadata> + <audience experiencelevel="expert" job="Customizing" type="Coder"/> + <category>Metadata</category> + <keywords> + <keyword>Tika</keyword> + <keyword>Content</keyword> + </keywords> + <prodinfo> + <prodname>Apache Tika</prodname> + <vrmlist> + <vrm version="1.x" release="Final" modification="2011/11/11"/> + </vrmlist> + </prodinfo> + </metadata> + </prolog> + <taskbody> + <context> + <p>The Apache Tika toolkit detects and extracts metadata and structured text content from various documents using existing parser libraries. You can find the latest release on the download page. See the Getting Started guide for instructions on how to start using Tika.</p> + + <p>Tika is a project of the Apache Software Foundation, and was formerly a subproject of Apache Lucene.</p> + </context> + </taskbody> +</task> http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEMLX.emlx ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEMLX.emlx b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEMLX.emlx new file mode 100644 index 0000000..d9a7126 --- /dev/null +++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEMLX.emlx @@ -0,0 +1,55 @@ +1795 +From: "Julien Nioche (JIRA)" <[email protected]> +To: [email protected] +Subject: [jira] Commented: (TIKA-461) RFC822 messages not parsed +Reply-To: [email protected] +Delivered-To: mailing list [email protected] +Date: Mon, 6 Sep 2010 05:25:34 -0400 (EDT) +In-Reply-To: <6089099.260231278600349994.JavaMail.jira@thor> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: 7bit +X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 +X-Virus-Checked: Checked by ClamAV on apache.org + + + [ https://issues.apache.org/jira/browse/TIKA-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906468#action_12906468 ] + +Julien Nioche commented on TIKA-461: +------------------------------------ + +I'll have a look at mime4j and try to use it in Tika + +> RFC822 messages not parsed +> -------------------------- +> +> Key: TIKA-461 +> URL: https://issues.apache.org/jira/browse/TIKA-461 +> Project: Tika +> Issue Type: Bug +> Components: parser +> Affects Versions: 0.7 +> Reporter: Joshua Turner +> Assignee: Julien Nioche +> +> Presented with an RFC822 message exported from Thunderbird, AutodetectParser produces an empty body, and a Metadata containing only one key-value pair: "Content-Type=message/rfc822". Directly calling MboxParser likewise gives an empty body, but with two metadata pairs: "Content-Encoding=us-ascii Content-Type=application/mbox". +> A quick peek at the source of MboxParser shows that the implementation is pretty naive. If the wiring can be sorted out, something like Apache James' mime4j might be a better bet. + +-- +This message is automatically generated by JIRA. +- +You can reply to this email to add a comment to the issue online. + +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd";> +<plist version="1.0"> +<dict> + <key>flags</key> + <integer>0</integer> + <key>sender</key> + <string>"Julien Nioche (JIRA)" <[email protected]></string> + <key>subject</key> + <string>[jira] Commented: (TIKA-461) RFC822 messages not parsed</string> + <key>to</key> + <string>[email protected]</string></dict> +</plist> http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xls ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xls b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xls new file mode 100644 index 0000000..86b2916 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xls differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xlsx ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xlsx b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xlsx new file mode 100644 index 0000000..8d5169f Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xlsx differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.flac ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.flac b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.flac new file mode 100644 index 0000000..ccec947 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.flac differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.oga ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.oga b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.oga new file mode 100644 index 0000000..37a1247 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.oga differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLV.flv ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLV.flv b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLV.flv new file mode 100644 index 0000000..d35e9bb Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLV.flv differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testGIF.gif ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testGIF.gif b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testGIF.gif new file mode 100644 index 0000000..e09e641 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testGIF.gif differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJAR.jar ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJAR.jar b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJAR.jar new file mode 100644 index 0000000..4677a62 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJAR.jar differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg new file mode 100644 index 0000000..1b93e77 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.gz ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.gz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.gz new file mode 100644 index 0000000..2ee8e9c Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.gz differ http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.tar.gz ---------------------------------------------------------------------- diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.tar.gz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.tar.gz new file mode 100644 index 0000000..3f35102 Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.tar.gz differ
