jiangxt2 opened a new pull request, #57328:
URL: https://github.com/apache/spark/pull/57328
Spark SQL has no built-in functions for IPv4 address manipulation, requiring
UDFs for common network log analysis tasks.
### What changes were proposed in this pull request?
Add three built-in SQL functions for IPv4 address manipulation:
- `inet_aton(str)`: converts an IPv4 address string to a 32-bit
integer. Returns null on invalid input in non-ANSI mode, throws
`INVALID_IPV4_ADDRESS` in ANSI mode.
- `inet_ntoa(value)`: converts a 32-bit integer to an IPv4 address
string in dotted-decimal notation. Returns null if the value is out
of range in non-ANSI mode, throws `INVALID_IPV4_LONG` in ANSI mode.
- `try_inet_aton(str)`: same as `inet_aton` but returns null on
invalid input regardless of ANSI mode, using the standard
`RuntimeReplaceable` + `InheritAnalysisRules` pattern.
Short-form IP addresses (1-3 dot-separated segments) are supported per
Flink convention. Leading zeros are parsed as decimal (e.g.
`'010.000.000.001'` -> `167772161`), matching MySQL and Flink, not
octal like POSIX `inet_aton(3)`.
The implementation uses pure integer arithmetic via `Ipv4Utils` (no
`java.net.InetAddress`), with whole-stage codegen support.
### Why are the changes needed?
Spark has no built-in IP address functions, while MySQL, ClickHouse,
Doris, Databend, VoltDB, and SingleStore all provide equivalents.
Common use cases include network log analysis (range queries on IP
prefixes), security audit, and IP geolocation, all of which currently
require UDFs.
```sql
-- Convert IP to integer for range queries
SELECT inet_aton('192.168.1.1'); -- 3232235777
-- Convert back to dotted-decimal
SELECT inet_ntoa(3232235777); -- 192.168.1.1
-- Null-safe version for dirty data
SELECT try_inet_aton('not-an-ip'); -- NULL
```
### Does this PR introduce _any_ user-facing change?
Yes. Three new SQL functions: `inet_aton`, `inet_ntoa`, `try_inet_aton`,
with corresponding Scala/Python DataFrame APIs. Two new error classes:
`INVALID_IPV4_ADDRESS`, `INVALID_IPV4_LONG`.
### How was this patch tested?
- `IpExpressionsSuite`: 22 test cases (valid/invalid inputs, short
forms, leading zeros, ANSI mode, round-trip, type mismatch, both
interpreted and codegen paths)
- SQL golden file tests: `ipv4-functions.sql`
- Spark Connect: 3 plan generation test cases + 3 explain golden files
- ScalaStyle: 0 errors
### Was this patch authored or co-authored using generative AI tooling?
Generative AI tooling (Claude Code) was used as an assistive tool for
implementation guidance and code review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
