Hi,
One step closer. Drill now seems to be communicating with at least the
Tachyon client library, picked up by using "tachyon:///" configuration, and
jets3t library is loaded.
The strange thing, at this stage, is that the Tachyon client seems to be
using the jets3t librarry and is complaining about missing s3 config.
I would think it enough for the Tachyon client to talk to the Tachyon
worker that is already handling all S3 communications just fine.
Anyways; if anyone here has been down this hole then please share.
Regards,
-Stefan
On Sun, Jul 26, 2015 at 3:22 PM, Stefán Baxter <ste...@activitystream.com>
wrote:
> Thank you.
>
> After a bit more digging I see that Tachyon is not HDFS compatible
> (explains the "tachyon://" connection prefix) and I doubt that the it will
> work at all until Drill starts officially supporting it.
>
> I will do this and report any positive progress.
>
> It's a shame that it does not work as Tachyon+S3 could be a really great
> Drill setup where frequently used files are available "locally" and S3 is
> used more like a deep storage than anything else.
>
> Regards,
> -Stefán
>
>
>
> On Sun, Jul 26, 2015 at 3:11 PM, David Tucker <dtuc...@maprtech.com>
> wrote:
>
>> One quick hint :
>> The file $DRILL_HOME/bin/hadoop-excludes.txt has a list of jars that ARE
>> NOT loaded during the bootstrap of Drill … and jets3t is one of them.
>> Commenting out the jets3t line in that file and restarting the drill bits
>> will at least get you past the first java dependency problem.
>>
>> — David
>>
>> > On Jul 26, 2015, at 4:52 AM, Stefán Baxter <ste...@activitystream.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > I'm trying to run Drill with Tachyon on top of S3.
>> >
>> > My Drill-Source config looks like this:
>> > {
>> > "type": "file",
>> > "enabled": true,
>> > "connection": "tachyon://localhost:19998/",
>> > "workspaces": {
>> > "root": {
>> > "location": "/",
>> > "writable": true,
>> > "defaultInputFormat": null
>> > }
>> > },
>> > "formats": {
>> > "psv": {
>> > "type": "text",
>> > "extensions": [
>> > "tbl"
>> > ],
>> > "delimiter": "|"
>> > },
>> > "csv": {
>> > "type": "text",
>> > "extensions": [
>> > "csv"
>> > ],
>> > "delimiter": ","
>> > },
>> > "tsv": {
>> > "type": "text",
>> > "extensions": [
>> > "tsv"
>> > ],
>> > "delimiter": "\t"
>> > },
>> > "parquet": {
>> > "type": "parquet"
>> > },
>> > "json": {
>> > "type": "json"
>> > },
>> > "avro": {
>> > "type": "avro"
>> > }
>> > }
>> > }
>> >
>> > Initilly I get:
>> > Error: SYSTEM ERROR: IOException: No FileSystem for scheme: tachyon
>> >
>> > When I copy the tachyon client (tachyon-client-0.7.0.jar) to the
>> > jar/3rdparty the system complains about missing tependencies (jets3).
>> >
>> > If I copy the full Tachyon client
>> > (tachyon-client-0.7.0-jar-with-dependencies.jar) to the jar/3rds party
>> the
>> > system will not start.
>> >
>> > Exception in thread "main" java.lang.ExceptionInInitializerError
>> > at org.apache.log4j.LogManager.getLogger(LogManager.java:44)
>> > at
>> org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:64)
>> > at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:277)
>> > at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:288)
>> > at org.apache.drill.exec.server.Drillbit.<clinit>(Drillbit.java:58)
>> > Caused by: java.lang.IllegalStateException: Detected both
>> > log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting
>> > StackOverflowError. See also
>> > http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
>> > at
>> org.apache.log4j.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:51)
>> > ... 5 more
>> >
>> >
>> > Has anyone here done this already and is willing to share some info?
>> >
>> > Regards,
>> > -Stefán
>>
>>
>
