[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737342#comment-14737342
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/1100
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
> Fix For: 0.10
>
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14736603#comment-14736603
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1100#issuecomment-138863495
Will merge this and add the issue number to the commit.
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733776#comment-14733776
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1100#issuecomment-138313755
Thank you for opening a new pull request.
+1 to merge (@ merging committer: please rename the commit to include the
JIRA id ;) )
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733774#comment-14733774
]
ASF GitHub Bot commented on FLINK-2003:
---
GitHub user thvasilo opened a pull request:
https://github.com/apache/flink/pull/1100
[FLINK-2003] [docs] Building on some encrypted filesystems leads to "File
name too long" error
Replaces #690, adding docs instead of changing the pom files.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/thvasilo/flink encfs
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1100.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1100
commit 0e1d83a58047a4af5a991e2c30bc8fc1398139fb
Author: Theodore Vasiloudis
Date: 2015-09-07T14:25:58Z
Added instructions for encrypted filesystems
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560872#comment-14560872
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user thvasilo closed the pull request at:
https://github.com/apache/flink/pull/690
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560871#comment-14560871
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user thvasilo commented on the pull request:
https://github.com/apache/flink/pull/690#issuecomment-105890348
OK, I'll close this PR and open up another one with the doc fixes.
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553880#comment-14553880
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/690#issuecomment-104181351
I would actually say into both, the README and the how to contribute
section.
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550129#comment-14550129
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user thvasilo commented on the pull request:
https://github.com/apache/flink/pull/690#issuecomment-103417077
So @rmetzger, where do you think the note should go? In the project
documentation or the "How to contribute" section of the project website?
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550052#comment-14550052
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/690#issuecomment-103400415
Yeah, I think that this may impact the compatibility of our code with other
code.
A better approach would be IMHO to try and reduce the length of the
classnames by reducing the nesting of functions and lambdas in scala a bit.
Pull a few methods out rather than using n-level nested lambdas.
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550014#comment-14550014
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user thvasilo commented on a diff in the pull request:
https://github.com/apache/flink/pull/690#discussion_r30577392
--- Diff: flink-yarn/pom.xml ---
@@ -113,6 +113,10 @@ under the License.
-Xms128m
-Xmx512m
+
+ -Xmax-classfile-name
+ 128
--- End diff --
According to the [man pages for
EncFS](http://manpages.ubuntu.com/manpages/trusty/man1/encfs.1.html):
> One such limitation is filename length. If your underlying filesystem
limits you to N characters in a filename, then EncFS will limit you
to
approximately 3*(N-2)/4. For example if the host filesystem limits
to
256 characters, then EncFS will be limited to 190 character
filenames.
This is because encrypted filenames are always longer then plaintext
filenames.
So that would be <190 characters for ext4, but it depends on the
filesystem.
I get the feeling that this creates unneeded headaches. I could change this
PR into a docs one instead, and add a mention of the workaround in the docs as
you suggested.
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14548392#comment-14548392
]
ASF GitHub Bot commented on FLINK-2003:
---
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/690#discussion_r30530340
--- Diff: flink-yarn/pom.xml ---
@@ -113,6 +113,10 @@ under the License.
-Xms128m
-Xmx512m
+
+ -Xmax-classfile-name
+ 128
--- End diff --
Does your encrypted file system only support 128 characters?
We should try to set the value as high as possible, otherwise, we might run
into
[problems](http://scala-language.1934581.n4.nabble.com/Xmax-classfile-name-td4637574.html)
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14548053#comment-14548053
]
ASF GitHub Bot commented on FLINK-2003:
---
GitHub user thvasilo opened a pull request:
https://github.com/apache/flink/pull/690
[FLINK-2003] Building on some encrypted filesystems leads to "File name
too long" error
Added limit for length of class names for flink-yarn, to avoid "Filename
too long" errors on encrypted filesystems.
I'm not sure if there is a way to do this globally for the whole project
(tried doing the same in the project pom and I couldn't get it to work), and
what would the potential repercussions be of doing something like that.
PS. Fixed some whitespace errors as well.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/thvasilo/flink encfs-build
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/690.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #690
commit b55a1a2e2a6486485087685e8ac082c97de06fb2
Author: Theodore Vasiloudis
Date: 2015-05-18T14:08:12Z
Added limit for length of class names, to avoid "Filename too long" errors
on encrypted filesystems.
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542183#comment-14542183
]
Theodore Vasiloudis commented on FLINK-2003:
As a note, in my case this was caused by the flink-yarn package, so adding the
compilation arg to its pom fixed the problems.
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-2003) Building on some encrypted filesystems leads to "File name too long" error
[
https://issues.apache.org/jira/browse/FLINK-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539694#comment-14539694
]
Robert Metzger commented on FLINK-2003:
---
I think we also should put an entry to our FAQ and maybe to the README.
> Building on some encrypted filesystems leads to "File name too long" error
> --
>
> Key: FLINK-2003
> URL: https://issues.apache.org/jira/browse/FLINK-2003
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Reporter: Theodore Vasiloudis
>Priority: Minor
> Labels: build, starter
>
> The classnames generated from the build system can be too long.
> Creating too long filenames in some encrypted filesystems is not possible,
> including encfs which is what Ubuntu uses.
> This the same as this [Spark
> issue|https://issues.apache.org/jira/browse/SPARK-4820]
> The workaround (taken from the linked issue) is to add in Maven under the
> compile options:
> {code}
> + -Xmax-classfile-name
> + 128
> {code}
> And in SBT add:
> {code}
> +scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
