Samiflux commented on PR #8951: URL: https://github.com/apache/hop/pull/8951#issuecomment-6270758605
Hi Matt, I tested the PR on Windows with Spark `local[*]` and `HADOOP_HOME` unset, using the rebuilt JAR, but the pipeline still fails with this error: ```text java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset ``` <img width="1250" height="255" alt="image" src="https://github.com/user-attachments/assets/3dd65bfc-e064-488b-bb38-b03be0417a05" /> The Help buttons work correctly, though. I may have missed something in the setup, but I wanted to share my test results. ```text 2026/10/19 17:00:41 [Basic] filename - Prepared native Spark pipeline engine with run configuration 'spark-local' (master=local[*]) 2026/10/19 17:00:44 [Basic] filename - Created SparkSession version=4.1.3 master=local[*] (owned) 2026/10/19 17:00:47 [Error] filename - Error executing Spark pipeline 2026/10/19 17:00:47 [Error] filename - org.apache.hop.core.exception.HopException: 2026/10/19 17:00:47 [Error] filename - Error writing Spark Dataset to 'file:///C:/.../spark/output/' as csv 2026/10/19 17:00:47 [Error] filename - java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems 2026/10/19 17:00:47 [Error] filename - 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.spark.pipeline.handler.SparkFileIoSupport.writeDataset(SparkFileIoSupport.java:125) 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.spark.pipeline.handler.SparkFileOutputHandler.handleTransform(SparkFileOutputHandler.java:149) 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.spark.pipeline.HopPipelineMetaToSparkConverter.createDataset(HopPipelineMetaToSparkConverter.java:258) 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.spark.engines.SparkPipelineEngine.lambda$0(SparkPipelineEngine.java:342) 2026/10/19 17:00:47 [Error] filename - at java.base/java.lang.Thread.run(Thread.java:1583) 2026/10/19 17:00:47 [Error] filename - Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:789) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:298) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:314) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:1179) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:861) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:901) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:900) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:900) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:1047) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:356) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.setupJob(HadoopMapReduceCommitProtocol.scala:180) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.datasources.FileFormatWriter$.writeAndCommit(FileFormatWriter.scala:268) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeWrite(FileFormatWriter.scala:306) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:189) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:195) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:117) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:115) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:129) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$2(QueryExecution.scala:185) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$8(SQLExecution.scala:177) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.withSessionTagsApplied(SQLExecution.scala:285) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$7(SQLExecution.scala:139) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.JobArtifactSet$.withActiveJobArtifactState(JobArtifactSet.scala:94) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.artifact.ArtifactManager.$anonfun$withResources$1(ArtifactManager.scala:112) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.artifact.ArtifactManager.withClassLoaderIfNeeded(ArtifactManager.scala:106) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.artifact.ArtifactManager.withResources(ArtifactManager.scala:111) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$6(SQLExecution.scala:139) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:308) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$1(SQLExecution.scala:138) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:804) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId0(SQLExecution.scala:92) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:250) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$1(QueryExecution.scala:185) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:717) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryExecution$$eagerlyExecute$1(QueryExecution.scala:184) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$3.applyOrElse(QueryExecution.scala:201) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$3.applyOrElse(QueryExecution.scala:194) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:491) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:107) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:491) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:37) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:360) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:356) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:37) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:37) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:467) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:194) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.$anonfun$lazyCommandExecuted$1(QueryExecution.scala:155) 2026/10/19 17:00:47 [Error] filename - at scala.util.Try$.apply(Try.scala:217) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.util.Utils$.doTryWithCallerStacktrace(Utils.scala:1392) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.util.Utils$.getTryWithCallerStacktrace(Utils.scala:1453) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.util.LazyTry.get(LazyTry.scala:58) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:160) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:239) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.classic.DataFrameWriter.runCommand(DataFrameWriter.scala:592) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.classic.DataFrameWriter.save(DataFrameWriter.scala:115) 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.spark.pipeline.handler.SparkFileIoSupport.writeDataset(SparkFileIoSupport.java:119) 2026/10/19 17:00:47 [Error] filename - ... 4 more 2026/10/19 17:00:47 [Error] filename - Suppressed: org.apache.spark.util.Utils$OriginalTryStackTraceException: Full stacktrace of original doTryWithCallerStacktrace caller 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:789) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:298) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:314) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:1179) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:861) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:901) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:900) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:900) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:873) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:1047) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:356) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.setupJob(HadoopMapReduceCommitProtocol.scala:180) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.datasources.FileFormatWriter$.writeAndCommit(FileFormatWriter.scala:268) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeWrite(FileFormatWriter.scala:306) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:189) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:195) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:117) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:115) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:129) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$2(QueryExecution.scala:185) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$8(SQLExecution.scala:177) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.withSessionTagsApplied(SQLExecution.scala:285) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$7(SQLExecution.scala:139) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.JobArtifactSet$.withActiveJobArtifactState(JobArtifactSet.scala:94) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.artifact.ArtifactManager.$anonfun$withResources$1(ArtifactManager.scala:112) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.artifact.ArtifactManager.withClassLoaderIfNeeded(ArtifactManager.scala:106) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.artifact.ArtifactManager.withResources(ArtifactManager.scala:111) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$6(SQLExecution.scala:139) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:308) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$1(SQLExecution.scala:138) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:804) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId0(SQLExecution.scala:92) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:250) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.$anonfun$eagerlyExecuteCommands$1(QueryExecution.scala:185) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:717) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryExecution$$eagerlyExecute$1(QueryExecution.scala:184) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$3.applyOrElse(QueryExecution.scala:201) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$3.applyOrElse(QueryExecution.scala:194) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:491) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:107) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:491) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:37) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:360) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:356) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:37) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:37) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:467) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:194) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.execution.QueryExecution.$anonfun$lazyCommandExecuted$1(QueryExecution.scala:155) 2026/10/19 17:00:47 [Error] filename - at scala.util.Try$.apply(Try.scala:217) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.util.Utils$.doTryWithCallerStacktrace(Utils.scala:1392) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.util.LazyTry.tryT$lzycompute(LazyTry.scala:46) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.util.LazyTry.tryT(LazyTry.scala:46) 2026/10/19 17:00:47 [Error] filename - ... 10 more 2026/10/19 17:00:47 [Error] filename - Caused by: java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.fileNotFoundException(Shell.java:601) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.getHadoopHomeDir(Shell.java:622) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:645) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.<clinit>(Shell.java:742) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1733) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.security.SecurityUtil.setConfigurationInternal(SecurityUtil.java:106) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:95) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:311) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:299) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:586) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2466) 2026/10/19 17:00:47 [Error] filename - at scala.Option.getOrElse(Option.scala:201) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2466) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.SparkContext.<init>(SparkContext.scala:339) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:3076) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.classic.SparkSession$Builder.$anonfun$build$2(SparkSession.scala:1015) 2026/10/19 17:00:47 [Error] filename - at scala.Option.getOrElse(Option.scala:201) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.classic.SparkSession$Builder.build(SparkSession.scala:1006) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:1035) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:908) 2026/10/19 17:00:47 [Error] filename - at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:923) 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.spark.engines.SparkPipelineEngine.buildSparkSession(SparkPipelineEngine.java:1136) 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.spark.engines.SparkPipelineEngine.createSparkSession(SparkPipelineEngine.java:905) 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.spark.engines.SparkPipelineEngine.startThreads(SparkPipelineEngine.java:300) 2026/10/19 17:00:47 [Error] filename - at org.apache.hop.ui.hopgui.file.pipeline.HopGuiPipelineGraph.lambda$41(HopGuiPipelineGraph.java:6766) 2026/10/19 17:00:47 [Error] filename - ... 1 more 2026/10/19 17:00:47 [Error] filename - Caused by: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.checkHadoopHomeInner(Shell.java:521) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:492) 2026/10/19 17:00:47 [Error] filename - at org.apache.hadoop.util.Shell.<clinit>(Shell.java:569) 2026/10/19 17:00:47 [Error] filename - ... 23 more ``` Reading the CSV works, and the pipeline runs successfully when I disable the hop to Spark file output; the error occurs only when writing the output. <img width="1155" height="277" alt="image" src="https://github.com/user-attachments/assets/8ac0e212-adb0-4416-b905-fa40a4dfe99d" /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
