eugenegujing opened a new pull request, #9041:
URL: https://github.com/apache/texera/pull/9041
### What changes were proposed in this PR?
The Volcano Plot draws a plain scatter of effect size vs `-log10(p)`,
colored by effect size, which is already the x axis. Nothing shows where the
significance bar is or which points clear it. This PR adds three optional
properties, all off by default, so saved workflows render as before.
| Property | Type | Effect when set |
|---|---|---|
| Significance threshold | number, `0 < x <= 1` | dashed horizontal line at
`-log10(threshold)` |
| Effect-size threshold | number, `> 0` | dashed vertical lines at
`+threshold` and `-threshold` |
| Color by significance | boolean | with both thresholds set, points are
colored `Up` / `Down` / `Not significant` with a legend instead of the
continuous `RdBu` scale |
```
Before: px.scatter(color=<effect column>, color_continuous_scale="RdBu")
After: same by default
+ fig.add_hline(...) when a significance
threshold is set
+ fig.add_vline(+eff) / fig.add_vline(-eff) when an effect-size
threshold is set
+ color="Significance", 3-color map when the toggle is on
and both thresholds are set
```
Categories use strict comparisons on the selected p-value column: `Up` = `p
< sig` and `effect > eff`, `Down` = `p < sig` and `effect < -eff`, everything
else `Not significant`. The category is written to a column named
`Significance` (also the legend title); an input column of that name would be
shadowed inside the operator only. Colors are ColorBrewer RdBu end points
`#B2182B` / `#2166AC` plus grey `#9E9E9E`.
Validation: the JSON schema bounds above stop bad values in the property
panel. For hand-edited JSON the operator also ignores a threshold it cannot
draw (NaN, infinite, <= 0, or a p cut-off > 1) and falls back to the continuous
coloring if the toggle is on but a threshold is missing. A cut-off of exactly 1
is allowed and draws its line at y = 0. If the cut-off's `-log10` is beyond the
data range the line may fall outside the auto-ranged axis; extending the range
is left for a follow-up.
Independent of the label-fix (#9037) and hover-column (#9038) PRs; all three
touch neighbouring lines of `VolcanoPlotOpDesc.scala`, so whichever lands
second needs a small rebase.
Figures below are the operator's generated Python from `main` and from this
branch, run on the 430-gene example workflow from the issue, with thresholds
0.05 and 1.
| Before | After (lines only) | After (lines + Color by significance) |
|---|---|---|
| <img width="1398" height="849" alt="before-main"
src="https://github.com/user-attachments/assets/2dd6c551-04b7-41cc-988d-cb19180703cd";
/> | <img width="1398" height="849" alt="thresholds-after-lines-only"
src="https://github.com/user-attachments/assets/ab3f4d36-3c8f-4073-8f73-ea967bf19174";
/> | <img width="1398" height="849" alt="thresholds-after-colored"
src="https://github.com/user-attachments/assets/d714fcc7-09cc-4e3f-897d-0da61bb1f8a6";
/> |
### Any related issues, documentation, discussions?
Closes #9039
PR #8504 adds a `generateStandaloneCode()` path to this operator that copies
the current template. Whichever lands first, the other needs a follow-up.
### How was this PR tested?
New tests in `VolcanoPlotOpDescSpec`, written before the source change:
- defaults: no lines, continuous coloring; each threshold alone emits only
its own line(s); both with the toggle off keep the continuous coloring
- toggle on with both thresholds: strict comparisons, `np.select` column,
exact color map and category order, no `color_continuous_scale`
- toggle on with one or no threshold: continuous fallback
- zero / negative thresholds and a p cut-off above 1 emit no line of their
own; `Infinity`, `-Infinity`, `NaN` from JSON never reach the Python
- a cut-off of exactly 1 draws its line and still colors by category
- thresholds are emitted as valid Python floats, checked for `1e-300`,
`Double.MIN_VALUE` and `1e21` in every site they appear
- the generated JSON schema carries `exclusiveMinimum` / `maximum` on the
two thresholds
- JSON deserialization: numbers, numeric strings, null, blank, absent, and
the boolean toggle
- a workflow JSON saved before this change generates byte-for-byte the same
code as a default-constructed operator
```
sbt -Dsbt.server.forcestart=true "WorkflowOperator/testOnly
*VolcanoPlotOpDescSpec"
# Tests: succeeded 24, failed 0, canceled 0, ignored 0, pending 1 (the
pending one is pre-existing)
sbt -Dsbt.server.forcestart=true "WorkflowOperator/scalafmtAll"
"WorkflowOperator/Test/scalafmtAll"
sbt -Dsbt.server.forcestart=true "WorkflowOperator/scalafixAll --check"
# success
```
Smoke run outside Texera (pytexera stubbed, pandas + plotly 5.24.1) on the
430-gene mock table, 420 rows after the operator drops zero and missing
p-values:
| Configuration | Lines | Traces |
|---|---|---|
| defaults | none | 1 continuous trace, 420 points |
| 0.05 / 1, toggle off | hline y = 1.301, vlines x = ±1 | 1 continuous trace
|
| 0.05 / 1, toggle on | same | `Up` 35, `Down` 35, `Not significant` 350 |
| 0.05 only, toggle on | hline only | continuous (fallback) |
| 0.05 / -1, toggle on | hline only | continuous (fallback) |
| 2 / 1, toggle on | vlines only | continuous (fallback) |
| `"Infinity"` / `"NaN"` from JSON | none | continuous, no such token in the
code |
| 1e-300 / 1e21, toggle on | hline y = 300, vlines x = ±1e21 | `Not
significant` 420 |
| 1 / 1, toggle on | hline y = 0, vlines x = ±1 | `Up` 41, `Down` 38, `Not
significant` 341 |
The exponent-form dumps pass `python -m py_compile`.
### Was this PR authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code (claude-fable-5-1). Co-authored with Claude;
the author reviewed the change before submission.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
