[ https://issues.apache.org/jira/browse/IMPALA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656052#comment-16656052 ]
Sean Mackrory commented on IMPALA-7726: --------------------------------------- {quote}You tried with subprocess.Popen - did you try with any other methods of calling hdfs? For instance, you could just call out to a bash script.{quote} Yeah so in the last iteration on the patch I switched from subprocess.check_output to subprocess.Popen precisely to make it easier to get stderr as a separate stream. I looked at all the args and output. Using identical args and every variation I can imagine (and I've tried with and without trailing slashes on directory names, with files and directories, inside and outside trash), I get identical and correct behavior on ABFS, S3 and HDFS. > Drop with purge tests fail against ABFS due to trash misbehavior > ---------------------------------------------------------------- > > Key: IMPALA-7726 > URL: https://issues.apache.org/jira/browse/IMPALA-7726 > Project: IMPALA > Issue Type: Bug > Reporter: Sean Mackrory > Assignee: Sean Mackrory > Priority: Major > > In testing IMPALA-7681, I've seen test_drop_partition_with_purge and > test_drop_table_with_purge fail because of files not found in the trash are a > drop without purge. I've traced that functionality through Hive, which uses > Hadoop's Trash API, and traced through a bunch of scenarios in that API with > ABFS and I can't see it misbehaving in any way. It also should be pretty > FS-agnostic. I also suspected a bug in abfs_utils.py's exists() function, but > have not been able to find one. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org