LibenHailu opened a new issue, #41133:
URL: https://github.com/apache/beam/issues/41133
### What happened?
Several small bugs exist in the Apache Beam YAML Python SDK
(`apache_beam.yaml`):
1. **`yaml_mapping._as_callable` crashes with `AttributeError` on
non-dict/non-str expressions and fails to detect simple field projections**:
- `_as_callable` calls `expr.get('expression')` *before* checking `if not
isinstance(expr, dict): raise ValueError(f"Ambiguous expression type (perhaps
missing quoting?): {expr}")`. When an unquoted non-string scalar or list is
passed (e.g., `fields: {x: 42}`), it raises `AttributeError: 'int' object has
no attribute 'get'` instead of the intended `ValueError`.
- Immediately after `if isinstance(expr, str): expr = {'expression':
expr}`, it checks `if expr in original_fields:`. Because `expr` is already a
`dict` and `original_fields` is a `list[str]`, `expr in original_fields` is
always `False` instead of checking `expr.get('expression') in original_fields`.
- `expr.pop('output_type', None)` mutates the caller's dictionary in
place when a `dict` is passed.
2. **`yaml_transform.chain_as_composite` rejects valid explicit `output`
when the last transform has both `name` and `type`**:
- In `chain_as_composite`, `is_not_output_of_last_transform` uses `or`:
```python
('name' in new_transforms[-1] and value != new_transforms[-1]['name'])
or
('type' in new_transforms[-1] and value != new_transforms[-1]['type'])
```
When the last transform in a `chain` specifies both `name` and `type`
(where `name != type`), this condition evaluates to `True` for every possible
`value`, raising `ValueError: Explicit output ... of the chain transform is not
an output of the last transform`.
3. **Missing `f`-string prefixes in YAML error messages**:
- `yaml_mapping._ExtractWindowingInfo`: `raise TypeError('Fields must be
a mapping or iterable of strings, got {fields}')` outputs literal `{fields}`.
- `yaml_combine.PyJsYamlCombine`: `raise TypeError('Unknown CombineFn:
{fn_spec}')` outputs literal `{fn_spec}`.
- `json_utils.json_schema_to_beam_schema`: `raise ValueError('Missing
properties for {json_schema}.')` outputs literal `{json_schema}`.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [x] Component: Python SDK
- [x] Component: Beam YAML
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
