Samiflux commented on PR #8951:
URL: https://github.com/apache/hop/pull/8951#issuecomment-6270758605

   Hi Matt, I tested the PR on Windows with Spark `local[*]` and `HADOOP_HOME` 
unset, using the rebuilt JAR, but the pipeline still fails with this error:
   
   ```text
   java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset
   ```
   
   <img width="1250" height="255" alt="image" 
src="https://github.com/user-attachments/assets/3dd65bfc-e064-488b-bb38-b03be0417a05";
 />
   
   The Help buttons work correctly, though. I may have missed something in the 
setup, but I wanted to share my test results.
   
   ```text
   2026/10/19 17:00:41 [Basic]    filename - Prepared native Spark pipeline 
engine with run configuration 'spark-local' (master=local[*])
   2026/10/19 17:00:44 [Basic]    filename - Created SparkSession version=4.1.3 
master=local[*] (owned)
   2026/10/19 17:00:47 [Error]    filename - Error executing Spark pipeline
   2026/10/19 17:00:47 [Error]    filename - 
org.apache.hop.core.exception.HopException: 
   2026/10/19 17:00:47 [Error]    filename - Error writing Spark Dataset to 
'file:///C:/.../spark/output/' as csv
   2026/10/19 17:00:47 [Error]    filename - java.io.FileNotFoundException: 
java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see 
https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems
   2026/10/19 17:00:47 [Error]    filename - 
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.spark.pipeline.handler.SparkFileIoSupport.writeDataset(SparkFileIoSupport.java:125)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.spark.pipeline.handler.SparkFileOutputHandler.handleTransform(SparkFileOutputHandler.java:149)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.spark.pipeline.HopPipelineMetaToSparkConverter.createDataset(HopPipelineMetaToSparkConverter.java:258)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.spark.engines.SparkPipelineEngine.lambda$0(SparkPipelineEngine.java:342)
   2026/10/19 17:00:47 [Error]    filename -    at 
java.base/java.lang.Thread.run(Thread.java:1583)
   2026/10/19 17:00:47 [Error]    filename - Caused by: 
java.lang.RuntimeException: java.io.FileNotFoundException: 
java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see 
https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:789)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:298)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:314)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:1179)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:861)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:901)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:900)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:900)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:1047)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:356)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.setupJob(HadoopMapReduceCommitProtocol.scala:180)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.writeAndCommit(FileFormatWriter.scala:268)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeWrite(FileFormatWriter.scala:306)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:189)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:195)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:117)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:115)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:129)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$2(QueryExecution.scala:185)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$8(SQLExecution.scala:177)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.SQLExecution$.withSessionTagsApplied(SQLExecution.scala:285)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$7(SQLExecution.scala:139)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.JobArtifactSet$.withActiveJobArtifactState(JobArtifactSet.scala:94)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.artifact.ArtifactManager.$anonfun$withResources$1(ArtifactManager.scala:112)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.artifact.ArtifactManager.withClassLoaderIfNeeded(ArtifactManager.scala:106)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.artifact.ArtifactManager.withResources(ArtifactManager.scala:111)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$6(SQLExecution.scala:139)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:308)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$1(SQLExecution.scala:138)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:804)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId0(SQLExecution.scala:92)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:250)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$1(QueryExecution.scala:185)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:717)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryExecution$$eagerlyExecute$1(QueryExecution.scala:184)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$3.applyOrElse(QueryExecution.scala:201)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$3.applyOrElse(QueryExecution.scala:194)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:491)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:107)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:491)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:37)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:360)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:356)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:37)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:37)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:467)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:194)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution.$anonfun$lazyCommandExecuted$1(QueryExecution.scala:155)
   2026/10/19 17:00:47 [Error]    filename -    at 
scala.util.Try$.apply(Try.scala:217)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.util.Utils$.doTryWithCallerStacktrace(Utils.scala:1392)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.util.Utils$.getTryWithCallerStacktrace(Utils.scala:1453)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.util.LazyTry.get(LazyTry.scala:58)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:160)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:239)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.classic.DataFrameWriter.runCommand(DataFrameWriter.scala:592)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.classic.DataFrameWriter.save(DataFrameWriter.scala:115)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.spark.pipeline.handler.SparkFileIoSupport.writeDataset(SparkFileIoSupport.java:119)
   2026/10/19 17:00:47 [Error]    filename -    ... 4 more
   2026/10/19 17:00:47 [Error]    filename -    Suppressed: 
org.apache.spark.util.Utils$OriginalTryStackTraceException: Full stacktrace of 
original doTryWithCallerStacktrace caller
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:789)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:298)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:314)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:1179)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:861)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:901)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:900)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:900)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:1047)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:356)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.setupJob(HadoopMapReduceCommitProtocol.scala:180)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.writeAndCommit(FileFormatWriter.scala:268)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeWrite(FileFormatWriter.scala:306)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:189)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:195)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:117)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:115)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:129)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$2(QueryExecution.scala:185)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$8(SQLExecution.scala:177)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.SQLExecution$.withSessionTagsApplied(SQLExecution.scala:285)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$7(SQLExecution.scala:139)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.JobArtifactSet$.withActiveJobArtifactState(JobArtifactSet.scala:94)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.artifact.ArtifactManager.$anonfun$withResources$1(ArtifactManager.scala:112)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.artifact.ArtifactManager.withClassLoaderIfNeeded(ArtifactManager.scala:106)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.artifact.ArtifactManager.withResources(ArtifactManager.scala:111)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$6(SQLExecution.scala:139)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:308)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$1(SQLExecution.scala:138)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:804)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId0(SQLExecution.scala:92)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:250)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$1(QueryExecution.scala:185)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:717)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryExecution$$eagerlyExecute$1(QueryExecution.scala:184)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$3.applyOrElse(QueryExecution.scala:201)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$3.applyOrElse(QueryExecution.scala:194)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:491)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:107)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:491)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:37)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:360)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:356)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:37)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:37)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:467)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:194)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.sql.execution.QueryExecution.$anonfun$lazyCommandExecuted$1(QueryExecution.scala:155)
   2026/10/19 17:00:47 [Error]    filename -            at 
scala.util.Try$.apply(Try.scala:217)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.util.Utils$.doTryWithCallerStacktrace(Utils.scala:1392)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.util.LazyTry.tryT$lzycompute(LazyTry.scala:46)
   2026/10/19 17:00:47 [Error]    filename -            at 
org.apache.spark.util.LazyTry.tryT(LazyTry.scala:46)
   2026/10/19 17:00:47 [Error]    filename -            ... 10 more
   2026/10/19 17:00:47 [Error]    filename - Caused by: 
java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and 
hadoop.home.dir are unset. -see 
https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.fileNotFoundException(Shell.java:601)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.getHadoopHomeDir(Shell.java:622)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:645)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.<clinit>(Shell.java:742)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1733)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.security.SecurityUtil.setConfigurationInternal(SecurityUtil.java:106)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:95)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:311)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:299)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:586)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2466)
   2026/10/19 17:00:47 [Error]    filename -    at 
scala.Option.getOrElse(Option.scala:201)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2466)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.SparkContext.<init>(SparkContext.scala:339)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:3076)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.classic.SparkSession$Builder.$anonfun$build$2(SparkSession.scala:1015)
   2026/10/19 17:00:47 [Error]    filename -    at 
scala.Option.getOrElse(Option.scala:201)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.classic.SparkSession$Builder.build(SparkSession.scala:1006)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:1035)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:908)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:923)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.spark.engines.SparkPipelineEngine.buildSparkSession(SparkPipelineEngine.java:1136)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.spark.engines.SparkPipelineEngine.createSparkSession(SparkPipelineEngine.java:905)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.spark.engines.SparkPipelineEngine.startThreads(SparkPipelineEngine.java:300)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hop.ui.hopgui.file.pipeline.HopGuiPipelineGraph.lambda$41(HopGuiPipelineGraph.java:6766)
   2026/10/19 17:00:47 [Error]    filename -    ... 1 more
   2026/10/19 17:00:47 [Error]    filename - Caused by: 
java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset.
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.checkHadoopHomeInner(Shell.java:521)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:492)
   2026/10/19 17:00:47 [Error]    filename -    at 
org.apache.hadoop.util.Shell.<clinit>(Shell.java:569)
   2026/10/19 17:00:47 [Error]    filename -    ... 23 more
   ``` 
   
   Reading the CSV works, and the pipeline runs successfully when I disable the 
hop to Spark file output; the error occurs only when writing the output.
   
   <img width="1155" height="277" alt="image" 
src="https://github.com/user-attachments/assets/8ac0e212-adb0-4416-b905-fa40a4dfe99d";
 />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to