szaszm commented on a change in pull request #1152: URL: https://github.com/apache/nifi-minifi-cpp/pull/1152#discussion_r827730482
########## File path: extensions/procfs/ProcFsSerialization.h ########## @@ -0,0 +1,164 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once +#include <concepts> + +#include "CpuStat.h" +#include "DiskStat.h" +#include "MemInfo.h" +#include "NetDev.h" +#include "ProcessStat.h" +#include "utils/gsl.h" + +namespace org::apache::nifi::minifi::extensions::procfs { + +void SerializeCPUStatData(const CpuStatData& cpu_stat_data, + std::invocable<const char(&)[], const uint64_t> auto serializer) { + serializer("user time", cpu_stat_data.getUser().count()); + serializer("nice time", cpu_stat_data.getNice().count()); Review comment: I don't see the point of passing a string as a reference unless it's used to detect the extent of the literal. I would either pass them as ordinary `const char*` and rely on the optimizer a bit more, or parameterize the implementation on the extent, but the second is not easy to write correct concept checks for. In the second parameter type check, I don't think const makes a difference when passing by value. ########## File path: extensions/procfs/ProcFsSerialization.h ########## @@ -0,0 +1,163 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once +#include <concepts> + +#include "CpuStat.h" +#include "DiskStat.h" +#include "MemInfo.h" +#include "NetDev.h" +#include "ProcessStat.h" +#include "utils/gsl.h" + +namespace org::apache::nifi::minifi::extensions::procfs { + +void SerializeCPUStatData(const CpuStatData& cpu_stat_data, + std::invocable<const char(&)[], const uint64_t> auto serializer) { + serializer("user time", cpu_stat_data.getUser().count()); + serializer("nice time", cpu_stat_data.getNice().count()); + serializer("system time", cpu_stat_data.getSystem().count()); + serializer("idle time", cpu_stat_data.getIdle().count()); + serializer("io wait time", cpu_stat_data.getIoWait().count()); + serializer("irq time", cpu_stat_data.getIrq().count()); + serializer("soft irq time", cpu_stat_data.getSoftIrq().count()); + serializer("steal time", cpu_stat_data.getSteal().count()); + serializer("guest time", cpu_stat_data.getGuest().count()); + serializer("guest nice time", cpu_stat_data.getGuestNice().count()); +} + +void SerializeNormalizedCPUStat(const CpuStatData& cpu_stat_data, + std::invocable<const char(&)[], const double> auto serializer) { + gsl_Expects(cpu_stat_data.getTotal() > 0ms); + serializer("user time %", cpu_stat_data.getUser()/cpu_stat_data.getTotal()); + serializer("nice time %", cpu_stat_data.getNice()/cpu_stat_data.getTotal()); + serializer("system time %", cpu_stat_data.getSystem()/cpu_stat_data.getTotal()); + serializer("idle time %", cpu_stat_data.getIdle()/cpu_stat_data.getTotal()); + serializer("io wait time %", cpu_stat_data.getIoWait()/cpu_stat_data.getTotal()); + serializer("irq time %", cpu_stat_data.getIrq()/cpu_stat_data.getTotal()); + serializer("soft irq %", cpu_stat_data.getSoftIrq()/cpu_stat_data.getTotal()); + serializer("steal time %", cpu_stat_data.getSteal()/cpu_stat_data.getTotal()); + serializer("guest time %", cpu_stat_data.getGuest()/cpu_stat_data.getTotal()); + serializer("guest nice time %", cpu_stat_data.getGuestNice()/cpu_stat_data.getTotal()); +} + +void SerializeDiskStatData(const DiskStatData& disk_stat_data, + std::invocable<const char(&)[], const uint64_t> auto serializer) { + serializer("Major Device Number", disk_stat_data.getMajorDeviceNumber()); + serializer("Minor Device Number", disk_stat_data.getMinorDeviceNumber()); + serializer("Reads Completed", disk_stat_data.getReadsCompleted()); + serializer("Reads Merged", disk_stat_data.getReadsMerged()); + serializer("Sectors Read", disk_stat_data.getSectorsRead()); + serializer("Writes Completed", disk_stat_data.getWritesCompleted()); + serializer("Writes Merged", disk_stat_data.getWritesMerged()); + serializer("Sectors Written", disk_stat_data.getSectorsWritten()); + serializer("IO-s in progress", disk_stat_data.getIosInProgress()); +} + +void SerializeDiskStatDataPerSec(const DiskStatData& disk_stat_data, + const std::chrono::duration<double> duration, + std::invocable<const char(&)[], const double> auto serializer) { + gsl_Expects(duration > 0ms); + serializer("Major Device Number", disk_stat_data.getMajorDeviceNumber()); + serializer("Minor Device Number", disk_stat_data.getMinorDeviceNumber()); + serializer("Reads Completed/sec", disk_stat_data.getReadsCompleted()/duration.count()); + serializer("Reads Merged/sec", disk_stat_data.getReadsMerged()/duration.count()); + serializer("Sectors Read/sec", disk_stat_data.getSectorsRead()/duration.count()); + serializer("Writes Completed/sec", disk_stat_data.getWritesCompleted()/duration.count()); + serializer("Writes Merged/sec", disk_stat_data.getWritesMerged()/duration.count()); + serializer("Sectors Written/sec", disk_stat_data.getSectorsWritten()/duration.count()); + serializer("IO-s in progress", disk_stat_data.getIosInProgress()/duration.count()); +} + +void SerializeMemInfo(const MemInfo& mem_info, + std::invocable<const char(&)[], const uint64_t> auto serializer) { + serializer("MemTotal", mem_info.getTotalMemory()); + serializer("MemFree", mem_info.getFreeMemory()); + serializer("MemAvailable", mem_info.getAvailableMemory()); + serializer("SwapTotal", mem_info.getTotalSwap()); + serializer("SwapFree", mem_info.getFreeSwap()); +} + +void SerializeNetDevData(const NetDevData& net_dev_data, + std::invocable<const char(&)[], const uint64_t> auto serializer) { + serializer("Bytes Received", net_dev_data.getBytesReceived()); + serializer("Packets Received", net_dev_data.getPacketsReceived()); + serializer("Receive Errors", net_dev_data.getReceiveErrors()); + serializer("Receive Drop Errors", net_dev_data.getReceiveDropErrors()); + serializer("Receive Fifo Errors", net_dev_data.getReceiveFifoErrors()); + serializer("Receive Frame Errors", net_dev_data.getReceiveFrameErrors()); + serializer("Compressed Packets Received", net_dev_data.getCompressedPacketsReceived()); + serializer("Multicast Frames Received", net_dev_data.getMulticastFramesReceived()); + + serializer("Bytes Transmitted", net_dev_data.getBytesTransmitted()); + serializer("Packets Transmitted", net_dev_data.getPacketsTransmitted()); + serializer("Transmit errors", net_dev_data.getTransmitErrors()); + serializer("Transmit drop errors", net_dev_data.getTransmitDropErrors()); + serializer("Transmit fifo errors", net_dev_data.getTransmitFifoErrors()); + serializer("Transmit collisions", net_dev_data.getTransmitCollisions()); + serializer("Transmit carrier losses", net_dev_data.getTransmitCarrierLosses()); + serializer("Compressed Packets Transmitted", net_dev_data.getCompressedPacketsTransmitted()); +} + +void SerializeNetDevDataPerSec(const NetDevData& net_dev_data, + const std::chrono::duration<double> duration, + std::invocable<const char(&)[], const double> auto serializer) { + gsl_Expects(duration > 0ms); + serializer("Bytes Received/sec", net_dev_data.getBytesReceived()/duration.count()); + serializer("Packets Received/sec", net_dev_data.getPacketsReceived()/duration.count()); + serializer("Receive Errors/sec", net_dev_data.getReceiveErrors()/duration.count()); + serializer("Receive Drop Errors/sec", net_dev_data.getReceiveDropErrors()/duration.count()); + serializer("Receive Fifo Errors/sec", net_dev_data.getReceiveFifoErrors()/duration.count()); + serializer("Receive Frame Errors/sec", net_dev_data.getReceiveFrameErrors()/duration.count()); + serializer("Compressed Packets Received/sec", net_dev_data.getCompressedPacketsReceived()/duration.count()); + serializer("Multicast Frames Received/sec", net_dev_data.getMulticastFramesReceived()/duration.count()); + + serializer("Bytes Transmitted/sec", net_dev_data.getBytesTransmitted()/duration.count()); + serializer("Packets Transmitted/sec", net_dev_data.getPacketsTransmitted()/duration.count()); + serializer("Transmit errors/sec", net_dev_data.getTransmitErrors()/duration.count()); + serializer("Transmit drop errors/sec", net_dev_data.getTransmitDropErrors()/duration.count()); + serializer("Transmit fifo errors/sec", net_dev_data.getTransmitFifoErrors()/duration.count()); + serializer("Transmit collisions/sec", net_dev_data.getTransmitCollisions()/duration.count()); + serializer("Transmit carrier losses/sec", net_dev_data.getTransmitCarrierLosses()/duration.count()); + serializer("Compressed Packets Transmitted/sec", net_dev_data.getCompressedPacketsTransmitted()/duration.count()); +} + +void SerializeProcessStat(const ProcessStat& process_stat, + std::invocable<const char(&)[], const uint64_t> auto uint64_t_serializer, + std::invocable<const char(&)[], const std::string_view&> auto string_serializer) { + string_serializer("COMM", process_stat.getComm()); + uint64_t_serializer("RES", process_stat.getMemory()); + uint64_t_serializer("CPUTIME", process_stat.getCpuTime().count()); +} + +void SerializeNormalizedProcessStat(const ProcessStat& process_stat_start, + const ProcessStat& process_stat_end, + const std::chrono::duration<double> all_cpu_time, + std::invocable<const char(&)[], const uint64_t> auto uint64_t_serializer, + std::invocable<const char(&)[], const std::string_view&> auto string_serializer, + std::invocable<const char(&)[], const double> auto double_serializer) { Review comment: I think a single serializer with an overloaded or templated call operator would be nicer. You can use the widely used `overloaded` utility: ``` ########## File path: extensions/procfs/ProcessStat.cpp ########## @@ -0,0 +1,54 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "ProcessStat.h" +#include <sstream> + +namespace org::apache::nifi::minifi::extensions::procfs { + +std::optional<ProcessStatData> ProcessStatData::parseProcessStatFile(std::istream& stat_file) { + std::stringstream stat_stream; + copy(std::istreambuf_iterator<char>(stat_file), + std::istreambuf_iterator<char>(), + std::ostreambuf_iterator<char>(stat_stream)); + + ProcessStatData process_stat_data; + stat_stream >> process_stat_data.pid_; + + std::string stat_stream_string = stat_stream.str(); + size_t comm_start = stat_stream_string.find_first_of('('); + size_t comm_end = stat_stream_string.find_last_of(')'); + if (comm_start == std::string::npos || comm_end == std::string::npos) + return std::nullopt; + process_stat_data.comm_ = stat_stream_string.substr(comm_start+1, comm_end-comm_start-1); + stat_stream.seekg(comm_end+2); Review comment: I think this could be simplified. ```suggestion process_stat_data.comm_ = [&] { std::string comm; stat_stream >> comm; return comm.substr(1, comm.size() - 2); }(); ``` Assuming that the field can not be missing. ########## File path: extensions/procfs/ProcFsJsonSerialization.h ########## @@ -0,0 +1,179 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once + +#include <string> +#include <string_view> + +#include "ProcFsSerialization.h" +#include "rapidjson/stream.h" +#include "rapidjson/document.h" + +namespace org::apache::nifi::minifi::extensions::procfs { + +namespace details { +class UInt64Serializer { + public: + UInt64Serializer(rapidjson::Value& json, rapidjson::Document::AllocatorType& alloc) : + json_(json), alloc_(alloc) { + } + void operator()(const char(&key)[], const uint64_t value) { Review comment: I think references to arrays have worked since forever, but I only know about using them to template-deduce static array bounds. I, too, would remove it here and rely on `const char*` or `std::string_view`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
