I have checked the docs, forums and FAQs but I can't find a way to do what
I need.
We have SpringBoot+Camel RESTful service:
Camel Core 2.22.1, Spring Boot starter 2.22.1 under Spring Boot 2.0.3.
We have the following (slightly obfuscated for IP reasons) rest routes to
receive, process and return a file:
*RouteBuilder*.java
...
@Override
public void configure() throws Exception {
from("file:func/input")
.unmarshal().json(JsonLibrary.Jackson, Options.class)
.process(optionsToHeaders)
.process(functionEngineProcessor)
.to("file:func/output?tempPrefix=.inprogress/");
...
*FileReceiverRouteBuilder*.java
...
@Override
public void configure() throws Exception {
rest("/func")
.post("/process")
.consumes(MediaType.TEXT_PLAIN_VALUE)
.route().routeId("func-upload-route")
.setHeader(Exchange.FILE_NAME, () -> UUID.randomUUID().toString()) // use
a UUID for inprocess filename
.to("file:uploads")
.setBody((ex) -> buildOptions(ex)) // appends json as function.options
.marshal().json(JsonLibrary.Jackson, true)
.log(LoggingLevel.INFO, "function.options: ${in.body}")
.to("file:func/input")
.setBody((ex) -> new ProcessResponse((String)
ex.getIn().getHeader(Exchange.FILE_NAME)))
.marshal().json(JsonLibrary.Jackson, true);
rest("/func")
.get("status/{token}")
.route().routeId("get-generated-func-file")
.choice()
.when((ex) -> Paths.get("function/output", (String)
ex.getIn().getHeader("token")).toFile().exists())
.to("direct:func-return-file-content")
.otherwise()
.setHeader(Exchange.HTTP_RESPONSE_CODE, constant("202")); /// send 202
until file complete
from("direct:func-return-file-content")
.log("returning file content")
.setBody((ex) -> Paths.get("func/output", (String)
ex.getIn().getHeader("token")).toFile())
.setHeader(Exchange.CONTENT_TYPE, constant(MediaType.TEXT_PLAIN_VALUE));
...
The *OptionsToHeadersProcessor* puts a filename and the Config options into
json.
public class OptionsToHeadersProcessor implements Processor{
public static final String PROCESS_PROFILE = "PROCESS_PROFILE";
public static final String FILE_NAME = "FILE_NAME";
@Override
public void process(Exchange exchange) throws Exception {
Options options = exchange.getIn().getBody(Options.class);
exchange.getIn().setHeader(FILE_NAME, options.getInputFile());
exchange.getIn().setHeader(PROCESS_PROFILE, options.getConfiguration());
//as header
}
The processing is working fine, files are returning etc. We have log4j
performing log appending (rollingfile) for the service, also OK.
However, we now want to enhance by generating a statistical set of data
about each file processed.
Statistics required include: -
- process.start time, end time and duration,
- one or more method timings - also start, end dutration (from within
the main *functionEngineProcessor*)
- data about the composition of the input file (e.g. bytes/lines) in
total, and for specific sections within the file.
We also need to compile all those stats into a *single (json) text* - a
single line of json - before we emit to a file.
This last requirement is causing the most issue; I think you can see that
if I need to put, say start, end time and duration together to log them
(even using log4j), I need to persist the start time value across the
processors, so that my statistics processor can calculate the final end
time and duration.
I have tried using an intercept, just before the first endpoint definition
in *FileReceiverRouteBuilder*.java:
interceptSendToEndpoint("direct:func*")
.log(LoggingLevel.INFO, "intercepted send to output for metrics")
.process(metricsProcessor);
However *metricsProcessor* cannot retrieve anything from headers (not even
PROCESS_PROFILE or FILE_NAME) since apparently file endpints do not persist
properties or headers.
The "token" is retrievable, but that is tied to the client REST signature
and I cant extend that without a lot of disruption. I can derive a
filename from the token, so processor is able to read the input and output
files and gather some of the required metrics. I would rather use/derive
from FILE_NAME, if it were available.
I therefore am unsure how to store values into some persistent object for
retreival / use in the metrics processor, since passing properties and
headers between routes isn't available with File EndPoints.
My current reading seems to imply JMS is a way to achieve this? I'm new to
camel etc and I wanted to see if I'm missing something obvious.
We would rather not propose any downstream concatenation if possible.
Any help much appreciated!
--
*Thomas Humphries*
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
