GitHub user dosubot[bot] added a comment to the discussion: Compilation of query data for a POST request
<!-- Answer --> Here's a breakdown of Superset's post-processing operations with examples: ## **[Aggregate](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/aggregate.py#L28-L46)** — Group data and apply calculations **Use when:** You need to compute statistics (sum, avg, median, percentile) by category ```json { "operation": "aggregate", "options": { "groupby": ["country"], "aggregates": { "total_sales": {"column": "sales", "operator": "sum"}, "median_price": {"column": "price", "operator": "median"} } } } ``` Operators: `sum`, `mean`, `median`, `min`, `max`, `std`, `count`, `percentile` (with `{"q": 25}` option) --- ## **[Sort](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/sort.py#L26-L47)** — Order rows by columns **Use when:** You need data in a specific order for display or further processing ```json { "operation": "sort", "options": { "by": ["sales", "date"], "ascending": [false, true] } } ``` --- ## **[Pivot](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/pivot.py#L31-L109)** — Create cross-tabulation tables **Use when:** You want to reshape data with row groups and column groups (like Excel pivot tables) ```json { "operation": "pivot", "options": { "index": ["country"], "columns": ["year"], "aggregates": {"total": {"operator": "sum", "column": "sales"}}, "metric_fill_value": 0 } } ``` --- ## **[Rolling](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/rolling.py#L31-L101)** — Moving window calculations **Use when:** You need smoothed time series (moving averages, rolling sums) ```json { "operation": "rolling", "options": { "rolling_type": "mean", "columns": {"sales": "sales_7day_avg"}, "window": 7, "min_periods": 1 } } ``` Types: `mean`, `sum`, `min`, `max`, `std`, `median`, `quantile` --- ## **[Diff](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/diff.py#L28-L50)** — Calculate row differences **Use when:** You need period-over-period changes (daily change, week-over-week) ```json { "operation": "diff", "options": { "columns": {"sales": "sales_change"}, "periods": 1 } } ``` --- ## **[Cum](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/cum.py#L30-L58)** — Cumulative calculations **Use when:** You need running totals or cumulative metrics ```json { "operation": "cum", "options": { "operator": "sum", "columns": {"sales": "cumulative_sales"} } } ``` Operators: `sum`, `prod`, `min`, `max` --- ## **[Contribution](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/contribution.py#L32-L107)** — Calculate percentages of totals **Use when:** You need to show what % each row/column contributes to totals ```json { "operation": "contribution", "options": { "orientation": "column" } } ``` --- ## **[Resample](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/resample.py#L26-L58)** — Change time frequency **Use when:** Converting hourly data to daily, or filling gaps in time series ```json { "operation": "resample", "options": { "rule": "1D", "method": "ffill" } } ``` Methods: `ffill`, `bfill`, `mean`, `sum`, `linear` --- ## **[Flatten](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/flatten.py#L37-L105)** — Simplify hierarchical columns **Use when:** After pivot operations create multi-level column headers ```json { "operation": "flatten", "options": {"reset_index": true} } ``` --- **Chaining:** Operations execute in order, so you can combine them: ```json "post_processing": [ {"operation": "pivot", "options": {...}}, {"operation": "flatten", "options": {}}, {"operation": "sort", "options": {"by": "total", "ascending": false}} ] ``` <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/16629bd1-33ad-4209-ba43-1c69a8bb6fe5?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/16629bd1-33ad-4209-ba43-1c69a8bb6fe5?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/16629bd1-33ad-4209-ba43-1c69a8bb6fe5?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/16629bd1-33ad-4209-ba43-1c69a8bb6fe5?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/16629bd1-33ad-4209-ba43-1c69a8bb6fe5?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/16629bd1-33ad-4209-ba43-1c69a8bb6fe5?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/16629bd1-33ad-4209-ba43-1c69a8bb6fe5?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=Here%27s%20a%20breakdown%20of%20Superset%27s%20post-processing%20operations%20with%20examples%3A%0A%0A%23%23%20%2A%2A%5BAggregate%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/aggregate.py%23L28-L46%29%2A%2A%20%E2%80%94%20Group%20data%20and%20apply%20calculations%0A%2A%2AUse%20when%3A%2A%2A%20You%20need%20to%20compute%20statistics%20%28sum%2C%20avg%2C%20median%2C%20percentile%29%20by%20category%0A%60%60%60json%0A%7B%0A%20%20%22operation%22%3A%20%22aggregate%22%2C%0A%20%20%22options%22%3A%20%7B%0A%20%20%20%20%22groupby%22%3A%20%5B%22country%22%5D%2C%0A%20%20%20%20%22aggregates%22%3A%20%7B%0A%20%20%20%20%20%20%22total_sales%22%3A%20%7B%22column%22%3A%20%22sales%22%2C%20%22ope rator%22%3A%20%22sum%22%7D%2C%0A%20%20%20%20%20%20%22median_price%22%3A%20%7B%22column%22%3A%20%22price%22%2C%20%22operator%22%3A%20%22median%22%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A%60%60%60%0AOperators%3A%20%60sum%60%2C%20%60mean%60%2C%20%60median%60%2C%20%60min%60%2C%20%60max%60%2C%20%60std%60%2C%20%60count%60%2C%20%60percentile%60%20%28with%20%60%7B%22q%22%3A%2025%7D%60%20option%29%0A%0A---%0A%0A%23%23%20%2A%2A%5BSort%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/sort.py%23L26-L47%29%2A%2A%20%E2%80%94%20Order%20rows%20by%20columns%0A%2A%2AUse%20when%3A%2A%2A%20You%20need%20data%20in%20a%20specific%20order%20for%20display%20or%20further%20processing%0A%60%60%60json%0A%7B%0A%20%20%22operation%22%3A%20%22sort%22%2C%0A%20%20%22options%22%3A%20%7B%0A%20%20%20%20%22by%22%3A%20%5B%22sales%22%2C%20%22date%22%5D%2C%0A%20%20%20%20%22ascending%22%3A%20%5Bfalse%2C%20true%5D%0A%20%20%7D%0A%7D%0A%60%60%60%0A%0A ---%0A%0A%23%23%20%2A%2A%5BPivot%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/pivot.py%23L31-L109%29%2A%2A%20%E2%80%94%20Create%20cross-tabulation%20tables%0A%2A%2AUse%20when%3A%2A%2A%20You%20want%20to%20reshape%20data%20with%20row%20groups%20and%20column%20groups%20%28like%20Excel%20pivot%20tables%29%0A%60%60%60json%0A%7B%0A%20%20%22operation%22%3A%20%22pivot%22%2C%0A%20%20%22options%22%3A%20%7B%0A%20%20%20%20%22index%22%3A%20%5B%22country%22%5D%2C%0A%20%20%20%20%22columns%22%3A%20%5B%22year%22%5D%2C%0A%20%20%20%20%22aggregates%22%3A%20%7B%22total%22%3A%20%7B%22operator%22%3A%20%22sum%22%2C%20%22column%22%3A%20%22sales%22%7D%7D%2C%0A%20%20%20%20%22metric_fill_value%22%3A%200%0A%20%20%7D%0A%7D%0A%60%60%60%0A%0A---%0A%0A%23%23%20%2A%2A%5BRolling%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/rolling.py%23L31-L101%29%2A%2A%20%E2% 80%94%20Moving%20window%20calculations%0A%2A%2AUse%20when%3A%2A%2A%20You%20need%20smoothed%20time%20series%20%28moving%20averages%2C%20rolling%20sums%29%0A%60%60%60json%0A%7B%0A%20%20%22operation%22%3A%20%22rolling%22%2C%0A%20%20%22options%22%3A%20%7B%0A%20%20%20%20%22rolling_type%22%3A%20%22mean%22%2C%0A%20%20%20%20%22columns%22%3A%20%7B%22sales%22%3A%20%22sales_7day_avg%22%7D%2C%0A%20%20%20%20%22window%22%3A%207%2C%0A%20%20%20%20%22min_periods%22%3A%201%0A%20%20%7D%0A%7D%0A%60%60%60%0ATypes%3A%20%60mean%60%2C%20%60sum%60%2C%20%60min%60%2C%20%60max%60%2C%20%60std%60%2C%20%60median%60%2C%20%60quantile%60%0A%0A---%0A%0A%23%23%20%2A%2A%5BDiff%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/diff.py%23L28-L50%29%2A%2A%20%E2%80%94%20Calculate%20row%20differences%0A%2A%2AUse%20when%3A%2A%2A%20You%20need%20period-over-period%20changes%20%28daily%20change%2C%20week-over-week%29%0A%60%60%60json%0A%7B%0A%20%20%22oper ation%22%3A%20%22diff%22%2C%0A%20%20%22options%22%3A%20%7B%0A%20%20%20%20%22columns%22%3A%20%7B%22sales%22%3A%20%22sales_change%22%7D%2C%0A%20%20%20%20%22periods%22%3A%201%0A%20%20%7D%0A%7D%0A%60%60%60%0A%0A---%0A%0A%23%23%20%2A%2A%5BCum%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/cum.py%23L30-L58%29%2A%2A%20%E2%80%94%20Cumulative%20calculations%0A%2A%2AUse%20when%3A%2A%2A%20You%20need%20running%20totals%20or%20cumulative%20metrics%0A%60%60%60json%0A%7B%0A%20%20%22operation%22%3A%20%22cum%22%2C%0A%20%20%22options%22%3A%20%7B%0A%20%20%20%20%22operator%22%3A%20%22sum%22%2C%0A%20%20%20%20%22columns%22%3A%20%7B%22sales%22%3A%20%22cumulative_sales%22%7D%0A%20%20%7D%0A%7D%0A%60%60%60%0AOperators%3A%20%60sum%60%2C%20%60prod%60%2C%20%60min%60%2C%20%60max%60%0A%0A---%0A%0A%23%23%20%2A%2A%5BContribution%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_p ostprocessing/contribution.py%23L32-L107%29%2A%2A%20%E2%80%94%20Calculate%20percentages%20of%20totals%0A%2A%2AUse%20when%3A%2A%2A%20You%20need%20to%20show%20what%20%25%20each%20row/column%20contributes%20to%20totals%0A%60%60%60json%0A%7B%0A%20%20%22operation%22%3A%20%22contribution%22%2C%0A%20%20%22options%22%3A%20%7B%0A%20%20%20%20%22orientation%22%3A%20%22column%22%0A%20%20%7D%0A%7D%0A%60%60%60%0A%0A---%0A%0A%23%23%20%2A%2A%5BResample%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/resample.py%23L26-L58%29%2A%2A%20%E2%80%94%20Change%20time%20frequency%0A%2A%2AUse%20when%3A%2A%2A%20Converting%20hourly%20data%20to%20daily%2C%20or%20filling%20gaps%20in%20time%20series%0A%60%60%60json%0A%7B%0A%20%20%22operation%22%3A%20%22resample%22%2C%0A%20%20%22options%22%3A%20%7B%0A%20%20%20%20%22rule%22%3A%20%221D%22%2C%0A%20%20%20%20%22method%22%3A%20%22ffill%22%0A%20%20%7D%0A%7D%0A%60%60%60%0AMethods%3A%20%60ffill%60%2 C%20%60bfill%60%2C%20%60mean%60%2C%20%60sum%60%2C%20%60linear%60%0A%0A---%0A%0A%23%23%20%2A%2A%5BFlatten%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/pandas_postprocessing/flatten.py%23L37-L105%29%2A%2A%20%E2%80%94%20Simplify%20hierarchical%20columns%0A%2A%2AUse%20when%3A%2A%2A%20After%20pivot%20operations%20create%20multi-level%20column%20headers%0A%60%60%60json%0A%7B%0A%20%20%22operation%22%3A%20%22flatten%22%2C%0A%20%20%22options%22%3A%20%7B%22reset_index%22%3A%20true%7D%0A%7D%0A%60%60%60%0A%0A---%0A%0A%2A%2AChaining%3A%2A%2A%20Operations%20execute%20in%20order%2C%20so%20you%20can%20combine%20them%3A%0A%60%60%60json%0A%22post_processing%22%3A%20%5B%0A%20%20%7B%22operation%22%3A%20%22pivot%22%2C%20%22options%22%3A%20%7B...%7D%7D%2C%0A%20%20%7B%22operation%22%3A%20%22flatten%22%2C%20%22options%22%3A%20%7B%7D%7D%2C%0A%20%20%7B%22operation%22%3A%20%22sort%22%2C%20%22options%22%3A%20%7B%22by%22%3A%20%22total%22%2C%20%22ascendin g%22%3A%20false%7D%7D%0A%5D%0A%60%60%60) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/discussions/38011) GitHub link: https://github.com/apache/superset/discussions/38011#discussioncomment-15832256 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
