oscerd commented on code in PR #26445:
URL: https://github.com/apache/camel/pull/26445#discussion_r4013455316
##########
components/camel-opa/src/main/java/org/apache/camel/component/opa/OpaEndpoint.java:
##########
@@ -57,16 +60,40 @@ public OpaEndpoint(final String uri, final Component
component, final OpaConfigu
@Override
protected void doStart() throws Exception {
super.doStart();
- opaClient = configuration.getOpaClient() != null
- ? configuration.getOpaClient()
- :
OpaPolicyEvaluator.createClient(configuration.getServerUrl(),
configuration.getBearerToken());
- evaluator = new OpaPolicyEvaluator(
- opaClient, policyPath, configuration.getAllowKey(),
configuration.getIncludeHeaders(),
- configuration.getIncludeProperties(),
configuration.isIncludeBody(), configuration.isFailOpen());
+ if (WASM_MODE.equalsIgnoreCase(configuration.getEvaluationMode())) {
+ evaluator = createWasmEvaluator();
+ } else {
Review Comment:
Fixed. `doStart` now accepts only `rest` and `wasm` and throws otherwise, so
a typo fails when the endpoint is created instead of turning into a server call
that ignores `policyBundle`.
I kept the explicit `else` for `rest` rather than treating empty as `rest`:
`evaluationMode` defaults to `rest` in the configuration, so an empty value can
only come from something like `evaluationMode={{opa.mode}}` resolving to
nothing — which is exactly the case worth failing on rather than guessing at.
Documented in `opa-component.adoc` alongside `policyBundle`.
_Claude Code on behalf of @oscerd_
##########
components/camel-opa/src/main/java/org/apache/camel/component/opa/OpaWasmEvaluator.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.opa;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+import java.util.Map;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.styra.opa.wasm.OpaPolicy;
+import com.styra.opa.wasm.OpaPolicyPool;
+import org.apache.camel.CamelContext;
+import org.apache.camel.support.ResourceHelper;
+import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
+import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
+import org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;
+
+/**
+ * Evaluates the policy in-process from a WebAssembly bundle produced by
{@code opa build -t wasm}.
+ * <p/>
+ * No OPA server is involved, so there is no network hop and no unreachable
policy decision point - at the cost of the
+ * policy being a build-time artefact rather than something a server
distributes and updates.
+ */
+public class OpaWasmEvaluator extends OpaPolicyEvaluator implements
AutoCloseable {
+
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+ private static final String POLICY_WASM = "policy.wasm";
+
+ private final OpaPolicyPool pool;
+
+ public OpaWasmEvaluator(byte[] wasm, String entrypoint, int poolSize,
String policyPath, String allowKey,
+ String includeHeaders, String includeProperties,
boolean includeBody, boolean failOpen) {
+ super(policyPath, allowKey, includeHeaders, includeProperties,
includeBody, failOpen);
+ // OpaPolicy carries mutable input/data and is not thread-safe, while
a Camel producer is invoked
+ // concurrently - so each exchange borrows its own instance rather
than sharing one
+ this.pool = OpaPolicyPool.create(() ->
OpaPolicy.builder().withPolicy(wasm).build().entrypoint(entrypoint),
Review Comment:
Good catch — confirmed, and fixed.
I reproduced it before changing anything, three borrows of the same instance
against `authz/decision`:
```
borrow #1 -> [{"result":{"allow":true,"reasons":[]}}] <- authz/decision
borrow #2 -> [{"result":true}] <- authz/allow,
entrypoint 0
borrow #3 -> [{"result":true}]
```
The entrypoint is now applied per borrow, in a `prepare()` that also
re-applies the bundle data (`reset()` clears that too — your other comment).
The javadoc on `prepare()` says why it cannot live in the factory.
On the test: `authz/decision` was being read from a *stale* `authz.wasm`.
The committed module predated the second `decision` rule in `authz.rego`, which
is why `failsClosedOnAnUndefinedDecisionJustLikeTheRestEngine` passed while
`OpaIT.keepsTheDenyReasonsFromADecisionObject` asserted the opposite against a
server running the same file. So I rebuilt both fixtures from the committed
source, added a rule that is genuinely undefined (`authz/strict_allow`, no
default) for the fail-closed test on both engines, and added
`keepsTheEntrypointAcrossPooledReuse` — five messages through `authz/decision`
with `poolSize=1`, asserting a decision object every time.
Reverting the fix turns five of the ten WASM tests red, including that one.
_Claude Code on behalf of @oscerd_
##########
components/camel-opa/src/main/java/org/apache/camel/component/opa/OpaWasmEvaluator.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.opa;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+import java.util.Map;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.styra.opa.wasm.OpaPolicy;
+import com.styra.opa.wasm.OpaPolicyPool;
+import org.apache.camel.CamelContext;
+import org.apache.camel.support.ResourceHelper;
+import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
+import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
+import org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;
+
+/**
+ * Evaluates the policy in-process from a WebAssembly bundle produced by
{@code opa build -t wasm}.
+ * <p/>
+ * No OPA server is involved, so there is no network hop and no unreachable
policy decision point - at the cost of the
+ * policy being a build-time artefact rather than something a server
distributes and updates.
+ */
+public class OpaWasmEvaluator extends OpaPolicyEvaluator implements
AutoCloseable {
+
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+ private static final String POLICY_WASM = "policy.wasm";
+
+ private final OpaPolicyPool pool;
+
+ public OpaWasmEvaluator(byte[] wasm, String entrypoint, int poolSize,
String policyPath, String allowKey,
+ String includeHeaders, String includeProperties,
boolean includeBody, boolean failOpen) {
+ super(policyPath, allowKey, includeHeaders, includeProperties,
includeBody, failOpen);
+ // OpaPolicy carries mutable input/data and is not thread-safe, while
a Camel producer is invoked
+ // concurrently - so each exchange borrows its own instance rather
than sharing one
+ this.pool = OpaPolicyPool.create(() ->
OpaPolicy.builder().withPolicy(wasm).build().entrypoint(entrypoint),
+ poolSize);
+ }
+
+ /**
+ * Loads a WebAssembly policy from a Camel resource location.
+ * <p/>
+ * {@code opa build} emits a {@code bundle.tar.gz} holding {@code
/policy.wasm} alongside the source and a manifest,
+ * so that is what an operator will actually have to hand; a bare {@code
.wasm} is accepted too.
+ */
+ public static byte[] loadPolicy(CamelContext camelContext, String
location) throws Exception {
+ try (InputStream in =
ResourceHelper.resolveMandatoryResourceAsInputStream(camelContext, location)) {
+ byte[] content = in.readAllBytes();
+ return isGzip(content) ? extractFromBundle(content, location) :
content;
+ }
+ }
+
+ private static boolean isGzip(byte[] content) {
+ return content.length > 1 && (content[0] & 0xff) == 0x1f &&
(content[1] & 0xff) == 0x8b;
+ }
+
+ private static byte[] extractFromBundle(byte[] bundle, String location)
throws Exception {
+ try (TarArchiveInputStream tar
+ = new TarArchiveInputStream(new GzipCompressorInputStream(new
ByteArrayInputStream(bundle)))) {
+ TarArchiveEntry entry;
+ while ((entry = tar.getNextEntry()) != null) {
+ if (!entry.isDirectory() &&
entry.getName().endsWith(POLICY_WASM)) {
Review Comment:
Fixed — loading it rather than documenting the limitation, since a policy
that reads `data.*` would otherwise deny everything with no indication why.
`loadPolicy` now returns a `Bundle` carrying both the module and the
bundle's `data.json`, and `prepare()` applies the data on every borrow for the
reason you note: `reset()` clears it along with the entrypoint.
Test fixture is `roles-bundle.tar.gz`, built from `wasm-data/` — a policy
deciding from `data.admins` with `{"admins": ["carol"]}` beside it.
`appliesTheDataDocumentPackedInTheBundle` sends several messages through a
single-instance pool and asserts carol is allowed and alice is not each time;
without the fix it fails on the first message.
Also documented, since the distinction still matters to an operator: a bare
`.wasm` carries the rules but not the data, and data a server would receive
through its Data API at runtime has no equivalent in `wasm` mode at all — that
case needs `evaluationMode=rest`.
_Claude Code on behalf of @oscerd_
##########
components/camel-opa/src/main/java/org/apache/camel/component/opa/OpaWasmEvaluator.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.opa;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+import java.util.Map;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.styra.opa.wasm.OpaPolicy;
+import com.styra.opa.wasm.OpaPolicyPool;
+import org.apache.camel.CamelContext;
+import org.apache.camel.support.ResourceHelper;
+import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
+import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
+import org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;
+
+/**
+ * Evaluates the policy in-process from a WebAssembly bundle produced by
{@code opa build -t wasm}.
+ * <p/>
+ * No OPA server is involved, so there is no network hop and no unreachable
policy decision point - at the cost of the
+ * policy being a build-time artefact rather than something a server
distributes and updates.
+ */
+public class OpaWasmEvaluator extends OpaPolicyEvaluator implements
AutoCloseable {
+
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+ private static final String POLICY_WASM = "policy.wasm";
+
+ private final OpaPolicyPool pool;
+
+ public OpaWasmEvaluator(byte[] wasm, String entrypoint, int poolSize,
String policyPath, String allowKey,
+ String includeHeaders, String includeProperties,
boolean includeBody, boolean failOpen) {
+ super(policyPath, allowKey, includeHeaders, includeProperties,
includeBody, failOpen);
+ // OpaPolicy carries mutable input/data and is not thread-safe, while
a Camel producer is invoked
+ // concurrently - so each exchange borrows its own instance rather
than sharing one
+ this.pool = OpaPolicyPool.create(() ->
OpaPolicy.builder().withPolicy(wasm).build().entrypoint(entrypoint),
+ poolSize);
+ }
+
+ /**
+ * Loads a WebAssembly policy from a Camel resource location.
+ * <p/>
+ * {@code opa build} emits a {@code bundle.tar.gz} holding {@code
/policy.wasm} alongside the source and a manifest,
+ * so that is what an operator will actually have to hand; a bare {@code
.wasm} is accepted too.
+ */
+ public static byte[] loadPolicy(CamelContext camelContext, String
location) throws Exception {
+ try (InputStream in =
ResourceHelper.resolveMandatoryResourceAsInputStream(camelContext, location)) {
+ byte[] content = in.readAllBytes();
+ return isGzip(content) ? extractFromBundle(content, location) :
content;
+ }
+ }
+
+ private static boolean isGzip(byte[] content) {
+ return content.length > 1 && (content[0] & 0xff) == 0x1f &&
(content[1] & 0xff) == 0x8b;
+ }
+
+ private static byte[] extractFromBundle(byte[] bundle, String location)
throws Exception {
+ try (TarArchiveInputStream tar
+ = new TarArchiveInputStream(new GzipCompressorInputStream(new
ByteArrayInputStream(bundle)))) {
+ TarArchiveEntry entry;
+ while ((entry = tar.getNextEntry()) != null) {
+ if (!entry.isDirectory() &&
entry.getName().endsWith(POLICY_WASM)) {
+ return tar.readAllBytes();
+ }
+ }
+ }
+ throw new IllegalArgumentException(
+ "No " + POLICY_WASM + " inside the bundle at " + location
+ + ". Build it with: opa build -t
wasm -e <entrypoint> <policy.rego>");
+ }
+
+ @Override
+ protected Object evaluateDecision(Map<String, Object> input) throws
Exception {
+ try (OpaPolicyPool.Loan loan = pool.borrow()) {
+ return
unwrap(loan.policy().evaluate(MAPPER.writeValueAsString(input)));
Review Comment:
Agreed, done. `evaluateDecision` now closes the loan on the success path and
calls `discard()` from the catch, so an instance that trapped mid-evaluation is
dropped rather than handed to the next exchange.
_Claude Code on behalf of @oscerd_
##########
components/camel-opa/src/main/java/org/apache/camel/component/opa/OpaEndpoint.java:
##########
@@ -57,16 +60,40 @@ public OpaEndpoint(final String uri, final Component
component, final OpaConfigu
@Override
protected void doStart() throws Exception {
super.doStart();
- opaClient = configuration.getOpaClient() != null
- ? configuration.getOpaClient()
- :
OpaPolicyEvaluator.createClient(configuration.getServerUrl(),
configuration.getBearerToken());
- evaluator = new OpaPolicyEvaluator(
- opaClient, policyPath, configuration.getAllowKey(),
configuration.getIncludeHeaders(),
- configuration.getIncludeProperties(),
configuration.isIncludeBody(), configuration.isFailOpen());
+ if (WASM_MODE.equalsIgnoreCase(configuration.getEvaluationMode())) {
Review Comment:
Done — see the reply on the other thread. Unknown modes now throw from
`doStart`.
_Claude Code on behalf of @oscerd_
##########
components/camel-opa/src/main/java/org/apache/camel/component/opa/OpaEndpoint.java:
##########
@@ -57,16 +60,40 @@ public OpaEndpoint(final String uri, final Component
component, final OpaConfigu
@Override
protected void doStart() throws Exception {
super.doStart();
- opaClient = configuration.getOpaClient() != null
- ? configuration.getOpaClient()
- :
OpaPolicyEvaluator.createClient(configuration.getServerUrl(),
configuration.getBearerToken());
- evaluator = new OpaPolicyEvaluator(
- opaClient, policyPath, configuration.getAllowKey(),
configuration.getIncludeHeaders(),
- configuration.getIncludeProperties(),
configuration.isIncludeBody(), configuration.isFailOpen());
+ if (WASM_MODE.equalsIgnoreCase(configuration.getEvaluationMode())) {
+ evaluator = createWasmEvaluator();
+ } else {
+ opaClient = configuration.getOpaClient() != null
+ ? configuration.getOpaClient()
+ :
OpaRestEvaluator.createClient(configuration.getServerUrl(),
configuration.getBearerToken());
+ evaluator = new OpaRestEvaluator(
+ opaClient, policyPath, configuration.getAllowKey(),
configuration.getIncludeHeaders(),
+ configuration.getIncludeProperties(),
configuration.isIncludeBody(), configuration.isFailOpen());
+ }
+ }
+
+ private OpaPolicyEvaluator createWasmEvaluator() throws Exception {
+ if (ObjectHelper.isEmpty(configuration.getPolicyBundle())) {
+ throw new IllegalArgumentException(
+ "policyBundle is required when evaluationMode=wasm; build
one with"
+ + " opa build -t wasm -e
<entrypoint> <policy.rego>");
+ }
+ // the entrypoint is fixed at build time and is not the same thing as
a data path, but opa build names it
+ // after the rule, so the policy path is the right default
+ String entrypoint =
ObjectHelper.isNotEmpty(configuration.getEntrypoint())
+ ? configuration.getEntrypoint() : policyPath;
+ byte[] wasm = OpaWasmEvaluator.loadPolicy(getCamelContext(),
configuration.getPolicyBundle());
Review Comment:
Done. The constructor now borrows one instance and runs `prepare()` on it,
so an ABI-incompatible or non-OPA module fails when the endpoint starts rather
than on the first message, and the first instance is warm.
Worth noting it made the tests stricter than I expected: because the warmup
consumes and returns an instance, the entrypoint bug from your other comment
now shows up on the *first* exchange rather than the second, so even the
single-message `authz/decision` tests go red without the per-borrow fix.
_Claude Code on behalf of @oscerd_
##########
components/camel-opa/pom.xml:
##########
@@ -47,6 +47,23 @@
<groupId>org.apache.camel</groupId>
<artifactId>camel-health</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.commons</groupId>
+ <artifactId>commons-compress</artifactId>
+ <version>${commons-compress-version}</version>
+ </dependency>
+ <dependency>
+ <groupId>com.styra.opa</groupId>
+ <artifactId>opa-java-wasm</artifactId>
+ <version>${opa-wasm-version}</version>
+ <exclusions>
+ <!-- the SDK declares a test engine at runtime scope -->
Review Comment:
You're right — dropped it. Checked the published POM: `junit-jupiter-engine`
(like `junit-jupiter-api`, `junit-jupiter-params`, `commons-io` and
`instancio-junit`) is declared at `test` scope, so it never reached our runtime
path and the exclusion only claimed otherwise.
_Claude Code on behalf of @oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
