This is an automated email from the ASF dual-hosted git repository.
hgruszecki pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iggy.git
The following commit(s) were added to refs/heads/master by this push:
new 83dbe5c4 chore(repo): add markdown linter to CI (#1879)
83dbe5c4 is described below
commit 83dbe5c4675938cef1a07c2c333ddf6c5780c933
Author: Hubert Gruszecki <[email protected]>
AuthorDate: Sat Jun 21 00:26:06 2025 +0200
chore(repo): add markdown linter to CI (#1879)
---
.github/workflows/ci-check-common.yml | 25 +++++++++++-
.github/workflows/ci-check-rust.yml | 1 +
.markdownlint.json | 3 ++
CONTRIBUTING.md | 4 +-
DEPENDENCIES.md | 2 +
README.md | 36 ++++++++---------
core/connectors/README.md | 15 +++----
core/connectors/sdk/README.md | 1 -
core/connectors/sinks/README.md | 1 -
core/connectors/sinks/quickwit_sink/README.md | 2 +-
core/connectors/sources/README.md | 2 +-
foreign/cpp/CONTRIBUTORS.md | 26 +++++++++++-
foreign/csharp/DEPENDENCIES.md | 4 +-
foreign/csharp/README.md | 58 ++++++++++++++++++++-------
foreign/go/samples/README.md | 4 +-
foreign/go/samples/consumer/README.md | 15 +++++--
foreign/go/samples/producer/README.md | 15 +++++--
foreign/java/README.md | 6 +--
foreign/node/CHANGELOG.md | 12 ++----
foreign/node/README.md | 30 +++++++-------
foreign/python/CONTRIBUTING.md | 22 +++++-----
foreign/python/README.md | 13 +++---
foreign/python/python_examples/README.md | 12 +++---
justfile | 5 ++-
scripts/licenses-list.sh | 19 +++++++--
25 files changed, 218 insertions(+), 115 deletions(-)
diff --git a/.github/workflows/ci-check-common.yml
b/.github/workflows/ci-check-common.yml
index 27546028..5f2f3ec2 100644
--- a/.github/workflows/ci-check-common.yml
+++ b/.github/workflows/ci-check-common.yml
@@ -20,8 +20,12 @@
#
# CI Check Common Workflow
#
-# This workflow validates commit messages and checks for conventional commits.
-# This workflow is mandatory and runs on pull request events.
+# This workflow runs common code quality checks that are not specific
+# to any programming language. Checks include:
+# - PR title validation using conventional commits
+# - license header verification
+# - license list validation
+# - markdown linting using markdownlint
#
name: ci-check-common
@@ -98,3 +102,20 @@ jobs:
rustup toolchain add --profile=minimal stable
rustup override set stable
- run: scripts/licenses-list.sh --check
+
+ markdownlint:
+ name: markdownlint
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout code
+ uses: actions/checkout@v4
+ - name: Setup Node.js
+ uses: actions/setup-node@v4
+ with:
+ node-version: '18'
+ - name: Install markdownlint-cli
+ run: |
+ npm install -g markdownlint-cli
+ - name: Run markdownlint
+ run: |
+ markdownlint '**/*.md' --ignore-path .gitignore
diff --git a/.github/workflows/ci-check-rust.yml
b/.github/workflows/ci-check-rust.yml
index 202a48c5..c12a2004 100644
--- a/.github/workflows/ci-check-rust.yml
+++ b/.github/workflows/ci-check-rust.yml
@@ -26,6 +26,7 @@
# - linting using `cargo clippy`
# - sorted dependencies check using `cargo sort`
# - documentation tests using `cargo test --doc`
+# - documentation generation using `cargo doc`
# - unused dependencies check using `cargo machete`
#
# This workflow can be triggered manually or by other workflows.
diff --git a/.markdownlint.json b/.markdownlint.json
index 1a958ff5..7885f5e4 100644
--- a/.markdownlint.json
+++ b/.markdownlint.json
@@ -2,5 +2,8 @@
"MD033": false,
"MD013": {
"line_length": 500
+ },
+ "MD024": {
+ "siblings_only": true
}
}
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index c4a81d30..31696169 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -63,7 +63,7 @@ For Linux or macOS, use the following command:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
-For Windows, download `rustup-init.exe` from
[here](https://win.rustup.rs/x86_64) instead.
+For Windows, download `rustup-init.exe` from [the Windows
installer](https://win.rustup.rs/x86_64) instead.
Rustup will read Iggy's `Cargo.toml` and set up everything else automatically.
To ensure that everything works correctly, run `cargo version` under Iggy's
root directory:
@@ -74,4 +74,4 @@ cargo 1.86.0 (adf9b6ad1 2025-02-28)
## How to build
-See [Quick
Start](https://github.com/apache/iggy?tab=readme-ov-file#quick-start)
\ No newline at end of file
+See [Quick
Start](https://github.com/apache/iggy?tab=readme-ov-file#quick-start)
diff --git a/DEPENDENCIES.md b/DEPENDENCIES.md
index 7c0ba0f3..6e87b891 100644
--- a/DEPENDENCIES.md
+++ b/DEPENDENCIES.md
@@ -1,3 +1,5 @@
+# Dependencies
+
actix-codec: 0.5.2, "Apache-2.0 OR MIT",
actix-cors: 0.7.1, "Apache-2.0 OR MIT",
actix-files: 0.6.6, "Apache-2.0 OR MIT",
diff --git a/README.md b/README.md
index cee31f65..5981bc68 100644
--- a/README.md
+++ b/README.md
@@ -171,49 +171,49 @@ Run the tests:
Start the server:
-`cargo run--bin iggy-server`
+`cargo run --bin iggy-server`
To quickly generate the sample data:
-`cargo run--bin data-seeder-tool`
+`cargo run --bin data-seeder-tool`
*Please note that all commands below are using `iggy` binary, which is part of
release (`cli` sub-crate).*
Create a stream with name `dev` (numerical ID will be assigned by server
automatically) using default credentials and `tcp` transport (available
transports: `quic`, `tcp`, `http`, default `tcp`):
-`cargo run--bin iggy -- --transport tcp --username iggy --password iggy stream
create dev`
+`cargo run --bin iggy -- --transport tcp --username iggy --password iggy
stream create dev`
List available streams:
-`cargo run--bin iggy -- --username iggy --password iggy stream list`
+`cargo run --bin iggy -- --username iggy --password iggy stream list`
Get `dev` stream details:
-`cargo run--bin iggy -- -u iggy -p iggy stream get dev`
+`cargo run --bin iggy -- -u iggy -p iggy stream get dev`
Create a topic named `sample` (numerical ID will be assigned by server
automatically) for stream `dev`, with 2 partitions (IDs 1 and 2), disabled
compression (`none`) and disabled message expiry (skipped optional parameter):
-`cargo run--bin iggy -- -u iggy -p iggy topic create dev sample 2 none`
+`cargo run --bin iggy -- -u iggy -p iggy topic create dev sample 2 none`
List available topics for stream `dev`:
-`cargo run--bin iggy -- -u iggy -p iggy topic list dev`
+`cargo run --bin iggy -- -u iggy -p iggy topic list dev`
Get topic details for topic `sample` in stream `dev`:
-`cargo run--bin iggy -- -u iggy -p iggy topic get dev sample`
+`cargo run --bin iggy -- -u iggy -p iggy topic get dev sample`
Send a message 'hello world' (message ID 1) to the stream `dev` to topic
`sample` and partition 1:
-`cargo run--bin iggy -- -u iggy -p iggy message send --partition-id 1 dev
sample "hello world"`
+`cargo run --bin iggy -- -u iggy -p iggy message send --partition-id 1 dev
sample "hello world"`
Send another message 'lorem ipsum' (message ID 2) to the same stream, topic
and partition:
-`cargo run--bin iggy -- -u iggy -p iggy message send --partition-id 1 dev
sample "lorem ipsum"`
+`cargo run --bin iggy -- -u iggy -p iggy message send --partition-id 1 dev
sample "lorem ipsum"`
Poll messages by a regular consumer with ID 1 from the stream `dev` for topic
`sample` and partition with ID 1, starting with offset 0, messages count 2,
without auto commit (storing consumer offset on server):
-`cargo run--bin iggy -- -u iggy -p iggy message poll --consumer 1 --offset 0
--message-count 2 --auto-commit dev sample 1`
+`cargo run --bin iggy -- -u iggy -p iggy message poll --consumer 1 --offset 0
--message-count 2 --auto-commit dev sample 1`
Finally, restart the server to see it is able to load the persisted data.
@@ -305,43 +305,43 @@ Then, run the benchmarking app with the desired options:
1. Sending (writing) benchmark
```bash
- cargo run--bin iggy-bench -r -- -v pinned-producer tcp
+ cargo run --bin iggy-bench -r -- -v pinned-producer tcp
```
2. Polling (reading) benchmark
```bash
- cargo run--bin iggy-bench -r -- -v pinned-consumer tcp
+ cargo run --bin iggy-bench -r -- -v pinned-consumer tcp
```
3. Parallel sending and polling benchmark
```bash
- cargo run--bin iggy-bench -r -- -v pinned-producer-and-consumer tcp
+ cargo run --bin iggy-bench -r -- -v pinned-producer-and-consumer tcp
```
4. Balanced sending to multiple partitions benchmark
```bash
- cargo run--bin iggy-bench -r -- -v balanced-producer tcp
+ cargo run --bin iggy-bench -r -- -v balanced-producer tcp
```
5. Consumer group polling benchmark:
```bash
- cargo run--bin iggy-bench -r -- -v balanced-consumer-group tcp
+ cargo run --bin iggy-bench -r -- -v balanced-consumer-group tcp
```
6. Parallel balanced sending and polling from consumer group benchmark:
```bash
- cargo run--bin iggy-bench -r -- -v balanced-producer-and-consumer-group tcp
+ cargo run --bin iggy-bench -r -- -v balanced-producer-and-consumer-group tcp
```
7. End to end producing and consuming benchmark (single task produces and
consumes messages in sequence):
```bash
- cargo run--bin iggy-bench -r -- -v end-to-end-producing-consumer tcp
+ cargo run --bin iggy-bench -r -- -v end-to-end-producing-consumer tcp
```
These benchmarks would start the server with the default configuration, create
a stream, topic and partition, and then send or poll the messages. The default
configuration is optimized for the best performance, so you might want to tweak
it for your needs. If you need more options, please refer to `iggy-bench`
subcommands `help` and `examples`.
diff --git a/core/connectors/README.md b/core/connectors/README.md
index 15abb705..f3784bb3 100644
--- a/core/connectors/README.md
+++ b/core/connectors/README.md
@@ -5,6 +5,7 @@ The highly performant and modular runtime for statically typed,
yet dynamically
**This is still WiP, and the runtime can be started only after compilation
from the source code (no installable package yet).**
## Features
+
- **High Performance**: Utilizes Rust's performance characteristics to ensure
fast data ingestion and egress.
- **Low memory footprint**: Designed with memory efficiency in mind,
minimizing the memory footprint of the connectors.
- **Modular Design**: Designed with modularity in mind, allowing for easy
extension and customization.
@@ -24,14 +25,14 @@ The highly performant and modular runtime for statically
typed, yet dynamically
4. Start the Iggy server and invoke the following commands via Iggy CLI to
create the example streams and topics used by the sample connectors.
-```
-iggy --username iggy --password iggy stream create example_stream
-iggy --username iggy --password iggy topic create example_stream example_topic
1 none 1d
-iggy --username iggy --password iggy stream create qw
-iggy --username iggy --password iggy topic create qw records 1 none 1d
-```
+ ```bash
+ iggy --username iggy --password iggy stream create example_stream
+ iggy --username iggy --password iggy topic create example_stream
example_topic 1 none 1d
+ iggy --username iggy --password iggy stream create qw
+ iggy --username iggy --password iggy topic create qw records 1 none 1d
+ ```
-5. Execute `cargo run--example sink-data-producer -r` which will start the
example data producer application, sending the messages to previously created
`qw` stream and `records` topic (this will be used by the Quickwit sink
connector).
+5. Execute `cargo run --example sink-data-producer -r` which will start the
example data producer application, sending the messages to previously created
`qw` stream and `records` topic (this will be used by the Quickwit sink
connector).
6. Start the connector runtime `cargo run --bin iggy-connectors -r` - you
should be able to browse Quickwit UI with records being constantly added to the
`events` index. At the same time, you should see the new messages being added
to the `example` stream and `topic1` topic by the test source connector - you
can use Iggy Web UI to browse the data. The messages will have applied the
basic fields transformations.
diff --git a/core/connectors/sdk/README.md b/core/connectors/sdk/README.md
index 13ee85fa..39c520a6 100644
--- a/core/connectors/sdk/README.md
+++ b/core/connectors/sdk/README.md
@@ -6,7 +6,6 @@ Moreover, it contains both, the `decoders` and `encoders`
modules, implementing
SDK is WiP, and it'd certainly benefit from having the support of multiple
format schemas, such as Protobuf, Avro, Flatbuffers etc. including
decoding/encoding the data between the different formats (when applicable) and
supporting the data transformations whenever possible (easy for JSON, but
complex for Bincode for example).
-
Last but not least, the different `transforms` are available, to transform
(add, update, delete etc.) the particular fields of the data being processed
via external configuration. It's as simple as adding a new transform to the
`transforms` section of the particular connector configuration:
```toml
diff --git a/core/connectors/sinks/README.md b/core/connectors/sinks/README.md
index 61c380be..0ab9b6a5 100644
--- a/core/connectors/sinks/README.md
+++ b/core/connectors/sinks/README.md
@@ -25,7 +25,6 @@ pub trait Sink: Send + Sync {
}
```
-
## Configuration
Sink is configured in the default `config` file used by runtime. Each sink
configuration, is part of the map of <String, SinkConfig>, which can be
represented using toml, json, or yaml.
diff --git a/core/connectors/sinks/quickwit_sink/README.md
b/core/connectors/sinks/quickwit_sink/README.md
index 40258aed..38dd4dc0 100644
--- a/core/connectors/sinks/quickwit_sink/README.md
+++ b/core/connectors/sinks/quickwit_sink/README.md
@@ -5,7 +5,7 @@ The Quickwit connector allows you to send data to the Quickwit
API using HTTP. T
## Configuration
- `url`: The URL of the Quickwit server.
-- `index`: The index configuration using YAML, as described
[here](https://quickwit.io/docs/configuration/index-config)
+- `index`: The index configuration using YAML, as described in the [Quickwit
index configuration docs](https://quickwit.io/docs/configuration/index-config)
```toml
[sinks.quickwit.config]
diff --git a/core/connectors/sources/README.md
b/core/connectors/sources/README.md
index 7fc6152a..b8472888 100644
--- a/core/connectors/sources/README.md
+++ b/core/connectors/sources/README.md
@@ -214,7 +214,7 @@ Eventually, compile the source code and update the runtime
configuration file us
And before starting the runtime, do not forget to create the specified stream
and topic e.g. via Iggy CLI.
-```
+```bash
iggy --username iggy --password iggy stream create example_stream
iggy --username iggy --password iggy topic create example_stream example_topic
1 none 1d
diff --git a/foreign/cpp/CONTRIBUTORS.md b/foreign/cpp/CONTRIBUTORS.md
index d3388d93..cadc1a69 100644
--- a/foreign/cpp/CONTRIBUTORS.md
+++ b/foreign/cpp/CONTRIBUTORS.md
@@ -2,11 +2,33 @@
## Introduction
-This is a collection of tips for developers of `iggy-cpp-client`. The initial
code is being built with the latest Visual Studio Code and its C++ dev
container, under `.devcontainer`. If you are using CLion, XCode or other
toolchains it may not have been tested, so please report issues and we will fix
them. We are also currently using [vcpkg](https://vcpkg.io) in [manifest
mode](https://learn.microsoft.com/en-us/vcpkg/users/manifests) for dependency
management. The appropriate packages ar [...]
+This is a collection of tips for developers of `iggy-cpp-client`. The initial
code is being
+built with the latest Visual Studio Code and its C++ dev container, under
`.devcontainer`.
+If you are using CLion, XCode or other toolchains it may not have been tested,
so please
+report issues and we will fix them. We are also currently using
[vcpkg](https://vcpkg.io)
+in [manifest mode](https://learn.microsoft.com/en-us/vcpkg/users/manifests)
for dependency
+management. The appropriate packages are downloaded and compiled locally as
part of the
+CMake build, but if you add a new dependency extra steps are required to
update the
+manifest and CMake configuration.
## Philosophy: code quality
-In VS Code the dev container automatically installs
[pre-commit](https://pre-commit.com), sets up the `.git` hooks directory and
installs all necessary tools for pre-commit checks. The relevant linters from
[super-linter](https://github.com/super-linter/super-linter) -- which runs in
CI automatically -- as well as the build, unit tests ad E2E tests are all run
for you on any changed files before any local Git commit, _before_ it gets
pushed. This is in line with the general project philo [...]
+In VS Code the dev container automatically installs
[pre-commit](https://pre-commit.com),
+sets up the `.git` hooks directory and installs all necessary tools for
pre-commit checks.
+The relevant linters from
[super-linter](https://github.com/super-linter/super-linter) --
+which runs in CI automatically -- as well as the build, unit tests ad E2E
tests are all
+run for you on any changed files before any local Git commit, _before_ it gets
pushed.
+This is in line with the general project philosophy to do as many quality
checks and fixes
+left-of-merge, meaning we aim to catch problems first in the IDE, then in
pre-commit checks,
+and finally in mandatory checks run in CI before the developer ever merges a
code change to
+the `main` branch. This is born of hard experience that quality problems, once
merged in,
+are much harder to repair. Not only is the problem out of context ("which of
the last 20
+commits caused this E2E test regression?") but accountability is lost: the
person who has
+broken a test, introduced a warning or violated coding standards _must_ be the
one to fix
+the problem. "Quality control is everyone's problem" is not enough: applied
literally,
+quality is nobody's problem, so this is better understood as a requirement
that everyone
+own the impact of the changes they themselves make to the codebase to the
maximum extent
+possible.
## Tested configurations
diff --git a/foreign/csharp/DEPENDENCIES.md b/foreign/csharp/DEPENDENCIES.md
index 6714cddb..bf37d349 100644
--- a/foreign/csharp/DEPENDENCIES.md
+++ b/foreign/csharp/DEPENDENCIES.md
@@ -1,3 +1,5 @@
+# Dependencies
+
FluentAssertions: "6.12.0", "Apache-2.0",
Microsoft.NET.Test.Sdk: "17.14.1", "MIT",
Moq: "4.20.72", "BSD-3-Clause",
@@ -7,4 +9,4 @@ xunit.runner.visualstudio: "3.1.0", "Apache-2.0",
coverlet.collector: "6.0.4", "MIT" ,
Microsoft.Extensions.Logging: "8.0.0", "MIT",
Microsoft.Extensions.Logging.Console:"7.0.0", "MIT"
-System.IO.Hashing: "9.0.5", "MIT",
\ No newline at end of file
+System.IO.Hashing: "9.0.5", "MIT",
diff --git a/foreign/csharp/README.md b/foreign/csharp/README.md
index 6ec0a954..fc8a6bc1 100644
--- a/foreign/csharp/README.md
+++ b/foreign/csharp/README.md
@@ -1,3 +1,5 @@
+# C# SDK for [Iggy](https://github.com/spetz/iggy)
+
<div align="center">
[](https://github.com/iggy-rs/iggy-dotnet-client/actions/workflows/dotnet.yml)
@@ -5,16 +7,15 @@
</div>
-# C# SDK for [Iggy](https://github.com/spetz/iggy)
-
-
-### Getting Started
+## Getting Started
Currently supported transfer protocols
-- TCP
-- HTTP
+
+- TCP
+- HTTP
The whole SDK revolves around `IIggyClient` interface to create an instance of
it, use following code
+
```c#
var loggerFactory = LoggerFactory.Create(builder =>
{
@@ -34,9 +35,11 @@ var bus = MessageStreamFactory.CreateMessageStream(options =>
};
}, loggerFactory);
```
+
Iggy necessitates the use of `ILoggerFactory` to generate logs from locations
that are inaccessible to the user.
In addition to the basic configuration settings, Iggy provides support for
batching send/poll messages at intervals, which effectively decreases the
frequency of network calls, this option is enabled by default.
+
```c#
//---Snip---
var bus = MessageStreamFactory.CreateMessageStream(options =>
@@ -64,8 +67,11 @@ var bus = MessageStreamFactory.CreateMessageStream(options =>
};
}, loggerFactory);
```
+
### Creating and logging in a user
+
To begin, utilize the root account (note that the root account cannot be
removed or updated).
+
```c#
var response = await bus.LoginUser(new LoginUserRequest
{
@@ -73,7 +79,9 @@ var response = await bus.LoginUser(new LoginUserRequest
Password = "iggy",
});
```
+
Furthermore, after logging in, you have the option to create an account with
customizable `Permissions`.
+
```c#
//---Snip---
await bus.CreateUser(new CreateUserRequest
@@ -131,7 +139,9 @@ var response = await bus.LoginUser(new LoginUserRequest
Password = "pa55w0rD!@",
});
```
+
Alternatively, once you've logged in, you can create a `Personal Access Token`
that can be reused for further logins.
+
```c#
var response = await bus.LoginUser(new LoginUserRequest
{
@@ -149,8 +159,11 @@ await bus.LoginWithPersonalAccessToken(new
LoginWithPersonalAccessToken
Token = patResponse.Token
});
```
+
### Creating first stream and topic
+
In order to create stream use `CreateStreamAsync` method.
+
```c#
await bus.CreateStreamAsync(new StreamRequest
{
@@ -158,8 +171,10 @@ await bus.CreateStreamAsync(new StreamRequest
Name = "first-stream",
});
```
+
Every stream has a topic to which you can broadcast messages, for the purpose
of create one
use `CreateTopicAsync` method.
+
```c#
var streamId = Identifier.Numeric(1);
await bus.CreateTopicAsync(streamId, new TopicRequest
@@ -169,11 +184,14 @@ await bus.CreateTopicAsync(streamId, new TopicRequest
TopicId = 1
});
```
+
Notice that both Stream aswell as Topic use `-` instead of space in its name,
Iggy will replace any spaces in
name with `-` instead, so keep that in mind.
### Sending messages
+
To send messages you can use `SendMessagesAsync` method.
+
```c#
Func<byte[], byte[]> encryptor = static payload =>
{
@@ -205,7 +223,9 @@ await bus.SendMessagesAsync(new MessageSendRequest
TopicId = topicId,
}, encryptor); //encryptor is optional
```
+
The `Message` struct has two fields `Id` and `Payload`.
+
```c#
struct Message
{
@@ -213,7 +233,9 @@ struct Message
public required byte[] Payload { get; init; }
}
```
+
Furthermore, there's a generic overload for this method that takes binary
serializer as argument.
+
```c#
//---Snip---
Func<Envelope, byte[]> serializer = static envelope =>
@@ -238,7 +260,9 @@ await bus.SendMessagesAsync(new MessageSendRequest<Envelope>
serializer,
encryptor);
```
+
Both generic and non generic method accept optional `Headers` dictionary.
+
```c#
//---Snip---
var headers = new Dictionary<HeaderKey, HeaderValue>
@@ -263,8 +287,11 @@ serializer,
encryptor,
headers);
```
+
### Fetching Messages
+
Fetching messages is done with `FetchMessagesAsync`.
+
```c#
Func<byte[], byte[]> decryptor = static payload =>
{
@@ -296,7 +323,9 @@ var messages = await bus.FetchMessagesAsync(new
MessageFetchRequest
},
decryptor);
```
+
Similarly, as with `SendMessagesAsync`, there's a generic overload that
accepts a binary deserializer.
+
```c#
//---Snip---
Func<byte[], Envelope> deserializer = serializedData =>
@@ -319,7 +348,9 @@ var messages = await bus.FetchMessagesAsync<Envelope>(new
MessageFetchRequest
AutoCommit = true
}, deserializer, decryptor);
```
+
Beyond the `FetchMessagesAsync` functionality, there's also a
`PollMessagesAsync` method that spawns new thread which polls messages in
background.
+
```c#
//---Snip---
await foreach (var messageResponse in bus.PollMessagesAsync<Envelope>(new
PollMessagesRequest
@@ -336,24 +367,23 @@ await foreach (var messageResponse in
bus.PollMessagesAsync<Envelope>(new PollMe
}
```
-It is worth noting that every method (except `PollMessagesAsync`) will throw
an `InvalidResponseException` when encountering an error.<br><br>
+
+It is worth noting that every method (except `PollMessagesAsync`) will throw
an `InvalidResponseException` when encountering an error.
+
If you register `IIggyClient` in a dependency injection container, you will
have access to interfaces
that encapsulate smaller parts of the system `IIggyStream` `IIggyTopic`
`IIggyPublisher` `IIggyConsumer` `IIggyConsumerGroup` `IIggyOffset`
`IIggyPartition` `IIggyUsers` `IIggyUtils`
For more information about how Iggy works check its
[documentation](https://docs.iggy.rs/)
-# Producer / Consumer Sample
+## Producer / Consumer Sample
-https://github.com/iggy-rs/iggy-dotnet-client/assets/112548209/3a89d2f5-d066-40d2-8b82-96c3e338007e
+<https://github.com/iggy-rs/iggy-dotnet-client/assets/112548209/3a89d2f5-d066-40d2-8b82-96c3e338007e>
-To run the samples, first get [Iggy](https://github.com/spetz/iggy), Run the
server with `cargo run--bin server`, then get the SDK, cd into `Iggy_SDK`
+To run the samples, first get [Iggy](https://github.com/spetz/iggy), Run the
server with `cargo run --bin iggy-server`, then get the SDK, cd into `Iggy_SDK`
and run following commands: `dotnet run -c Release --project
Iggy_Sample_Producer` for producer, `dotnet run -c Release --project
Iggy_Sample_Consumer`
for consumer.
## TODO
-- Add support for `ASP.NET Core` Dependency Injection
-
-
-
+- Add support for `ASP.NET Core` Dependency Injection
diff --git a/foreign/go/samples/README.md b/foreign/go/samples/README.md
index 6307f90b..c7b82f8f 100644
--- a/foreign/go/samples/README.md
+++ b/foreign/go/samples/README.md
@@ -1,6 +1,6 @@
-## Samples
+# Samples
Check out our sample projects below:
- <a
href="https://github.com/iggy-rs/iggy-go-client/blob/dev/samples/consumer/README.md";>Consumer</a>
- simple message consumer implementation.
-- <a
href="https://github.com/iggy-rs/iggy-go-client/blob/dev/samples/producer/README.md";>Producer</a>
- simple message producer implementation.
\ No newline at end of file
+- <a
href="https://github.com/iggy-rs/iggy-go-client/blob/dev/samples/producer/README.md";>Producer</a>
- simple message producer implementation.
diff --git a/foreign/go/samples/consumer/README.md
b/foreign/go/samples/consumer/README.md
index 1dcdf7ab..8350bb2a 100644
--- a/foreign/go/samples/consumer/README.md
+++ b/foreign/go/samples/consumer/README.md
@@ -1,4 +1,5 @@
-<!-- ABOUT THE PROJECT -->
+# Consumer Sample
+
## About The Project
Sample message consumer written using the `iggy-go` sdk.
@@ -16,17 +17,23 @@ In order to use this SDK you need to install golang on your
enviroment. Here's a
## Usage
In order to successfully launch the consumer app follow these steps:
+
1. Clone iggy repo and run it in background
+
```sh
git clone https://github.com/iggy-rs/iggy.git
cd iggy
- cargo run--bin server -r
+ cargo run --bin iggy-server -r
```
-1. Open new terminal instance and enter `iggy-go` root folder
+
+2. Open new terminal instance and enter `iggy-go` root folder
+
```sh
cd iggy-go
```
-2. Run the consumer
+
+3. Run the consumer
+
```sh
go run ./samples/consumer
```
diff --git a/foreign/go/samples/producer/README.md
b/foreign/go/samples/producer/README.md
index 432b1c64..2386594f 100644
--- a/foreign/go/samples/producer/README.md
+++ b/foreign/go/samples/producer/README.md
@@ -1,4 +1,5 @@
-<!-- ABOUT THE PROJECT -->
+# Producer Sample
+
## About The Project
Sample message producer written using the `iggy-go` sdk.
@@ -16,17 +17,23 @@ In order to use this SDK you need to install golang on your
enviroment. Here's a
## Usage
In order to successfully launch the producer app follow these steps:
+
1. Clone iggy repo and run it in background
+
```sh
git clone https://github.com/iggy-rs/iggy.git
cd iggy
- cargo run--bin server -r
+ cargo run --bin iggy-server -r
```
-1. Open new terminal instance and enter `iggy-go` root folder
+
+2. Open new terminal instance and enter `iggy-go` root folder
+
```sh
cd iggy-go
```
-2. Run the producer
+
+3. Run the producer
+
```sh
go run ./samples/producer
```
diff --git a/foreign/java/README.md b/foreign/java/README.md
index 0801dd4e..53388363 100644
--- a/foreign/java/README.md
+++ b/foreign/java/README.md
@@ -11,15 +11,15 @@ Official Java client SDK for [Apache
Iggy](https://iggy.apache.org) message stre
The client currently supports HTTP and TCP protocols with blocking
implementation.
-### Adding the client to your project
+## Adding the client to your project
Add dependency to `pom.xml` or `build.gradle` file.
You can find the latest version in Maven Central repository:
-https://central.sonatype.com/artifact/org.apache.iggy/iggy-java-sdk
+<https://central.sonatype.com/artifact/org.apache.iggy/iggy-java-sdk>
-### Implement consumer and producer
+## Implement consumer and producer
You can find examples for
simple
[consumer](https://github.com/apache/iggy/blob/master/foreign/java/examples/simple-consumer/src/main/java/org/apache/iggy/SimpleConsumer.java)
diff --git a/foreign/node/CHANGELOG.md b/foreign/node/CHANGELOG.md
index 42f099a4..732a4f58 100644
--- a/foreign/node/CHANGELOG.md
+++ b/foreign/node/CHANGELOG.md
@@ -1,5 +1,6 @@
-##
[1.0.6](https://github.com/iggy-rs/iggy-node-client/compare/v1.0.5...v1.0.6)
(2025-01-14)
+# Changelog
+##
[1.0.6](https://github.com/iggy-rs/iggy-node-client/compare/v1.0.5...v1.0.6)
(2025-01-14)
### Bug Fixes
@@ -7,41 +8,35 @@
##
[1.0.5](https://github.com/iggy-rs/iggy-node-client/compare/v1.0.4...v1.0.5)
(2025-01-14)
-
### Bug Fixes
* set package.json license to Apache-2.0 as well
([#24](https://github.com/iggy-rs/iggy-node-client/issues/24))
([2523c0f](https://github.com/iggy-rs/iggy-node-client/commit/2523c0fde82958e8a13bb95b2c2d0babe0d1d290))
##
[1.0.4](https://github.com/iggy-rs/iggy-node-client/compare/v1.0.3...v1.0.4)
(2024-12-16)
-
### Bug Fixes
* add tests, fix headers.double type, ehance typing
([431063f](https://github.com/iggy-rs/iggy-node-client/commit/431063f253e0fb1739188bbd6614cfc06ccb3fd4))
##
[1.0.3](https://github.com/iggy-rs/iggy-node-client/compare/v1.0.2...v1.0.3)
(2024-11-24)
-
### Bug Fixes
* upgrade dependencies
([#19](https://github.com/iggy-rs/iggy-node-client/issues/19))
([a05d5c2](https://github.com/iggy-rs/iggy-node-client/commit/a05d5c2484f8711f72a23c4ee1222d29e323a5ba))
##
[1.0.2](https://github.com/iggy-rs/iggy-node-client/compare/v1.0.1...v1.0.2)
(2024-11-24)
-
### Bug Fixes
* **npm:** configure package as public
([#18](https://github.com/iggy-rs/iggy-node-client/issues/18))
([7a56455](https://github.com/iggy-rs/iggy-node-client/commit/7a5645512a564322af343bc7ba748c8c58dfab1e))
##
[1.0.1](https://github.com/iggy-rs/iggy-node-client/compare/v1.0.0...v1.0.1)
(2024-11-24)
-
### Bug Fixes
* **npm:** publish package
([#17](https://github.com/iggy-rs/iggy-node-client/issues/17))
([6e2f60e](https://github.com/iggy-rs/iggy-node-client/commit/6e2f60e2f4484b57596285ff79d76ea647962595))
-# 1.0.0 (2024-11-24)
-
+## 1.0.0 (2024-11-24)
### Bug Fixes
@@ -74,7 +69,6 @@
* use debug lib, make poolsize configurable
([2f66e89](https://github.com/iggy-rs/iggy-node-client/commit/2f66e89220b30b3e33aa773b6471b1669658f37f))
* use pat as token for semantic release
([#10](https://github.com/iggy-rs/iggy-node-client/issues/10))
([5eccfd2](https://github.com/iggy-rs/iggy-node-client/commit/5eccfd281763773140ce9be39a2c022b2de803b6))
-
### Features
* add base ci workflow
([#2](https://github.com/iggy-rs/iggy-node-client/issues/2))
([4cbde14](https://github.com/iggy-rs/iggy-node-client/commit/4cbde140409841bf3d440f01ccaca1a855f37b13))
diff --git a/foreign/node/README.md b/foreign/node/README.md
index 575a387a..7f8f33b9 100644
--- a/foreign/node/README.md
+++ b/foreign/node/README.md
@@ -6,12 +6,12 @@ diclaimer: although all iggy commands & basic client/stream
are implemented this
note: This lib started as _iggy-bin_ (
[github](https://github.com/T1B0/iggy-bin) /
[npm](https://www.npmjs.com/package/iggy-bin)) before migrating under iggy-rs
org. package [email protected] is equivalent to @iggy.rs/[email protected] and migrating
again under apache iggy monorepo (
[github](https://github.com/apache/iggy/tree/master/foreign/node) and is now
published on npmjs as @apache-iggy/node-sdk
-note: previous works on node.js http client has been moved to
[iggy-node-http-client](<https://github.com/iggy-rs/iggy-node-http-client)
(moved on 04 July 2024)
+note: previous works on node.js http client has been moved to
[iggy-node-http-client](<https://github.com/iggy-rs/iggy-node-http-client>)
(moved on 04 July 2024)
## install
-```
-$ npm i @apache-iggy/node-sdk
+```bash
+npm i @apache-iggy/node-sdk
```
## basic usage
@@ -34,32 +34,34 @@ const stats = await c.system.getStats();
### Install
-```
-$ npm ci
+```bash
+npm ci
```
### build
-```
-$ npm run build
+```bash
+npm run build
```
### test
#### unit tests
-```
-$ npm run test
+
+```bash
+npm run test
```
#### e2e tests
-e2e test expect an iggy-server at tcp://127.0.0.1:8090
-```
-$ npm run test:e2e
+e2e test expect an iggy-server at tcp://127.0.0.1:8090
+
+```bash
+npm run test:e2e
```
### lint
-```
-$ npm run lint
+```bash
+npm run lint
```
diff --git a/foreign/python/CONTRIBUTING.md b/foreign/python/CONTRIBUTING.md
index 48511a9d..6ff9699e 100644
--- a/foreign/python/CONTRIBUTING.md
+++ b/foreign/python/CONTRIBUTING.md
@@ -4,45 +4,43 @@ This repository provides a Python library powered by Rust
using `pyo3`. It also
## Quick Start
-### 1. Run the Server with Docker:
+### 1. Run the Server with Docker
Ensure you have Docker installed on your system. Then, execute the following
command:
-```
+```bash
docker run --rm -p 8080:8080 -p 3000:3000 -p 8090:8090 iggyrs/iggy:0.4.21
```
-
This command runs the server and maps the specified ports to your local
machine.
-### 2. Install loguru:
+### 2. Install loguru
+
Loguru is advanced library used for logging information about procceses.
Install it with:
-```
+```bash
pip install loguru
```
-### 3. Install Maturin:
+### 3. Install Maturin
Maturin is used for building Rust binaries for Python. Install it with:
-```
+```bash
pip install maturin
```
-### 4. Build and Install the pyo3 Library:
+### 4. Build and Install the pyo3 Library
Navigate to your library's root directory and execute:
-```
+```bash
python -m venv .venv && source .venv/bin/activate
maturin develop
```
-
This will build the Rust library and make it available for Python.
-### 5. Running the Examples:
+### 5. Running the Examples
Go to [python_examples/README.md](python_examples/README.md) for instructions
on running the examples.
-
diff --git a/foreign/python/README.md b/foreign/python/README.md
index 9d18ed18..b368abd4 100644
--- a/foreign/python/README.md
+++ b/foreign/python/README.md
@@ -4,7 +4,6 @@
Apache Iggy is the persistent message streaming platform written in Rust,
supporting QUIC, TCP and HTTP transport protocols, capable of processing
millions of messages per second.
-
## Installation
To install `iggy`, use pip:
@@ -17,28 +16,28 @@ pip install iggy-py
- Python 3.7+
-## Usage and Examples:
+## Usage and Examples
All examples rely on a running iggy server. To start the server, execute:
-```
+```bash
docker run --rm -p 8080:8080 -p 3000:3000 -p 8090:8090 iggyrs/iggy:0.4.21
```
## Generating Stub Files
+
To generate a stub file, execute the following command:
-```
+```bash
cargo run --bin stub_gen
```
Refer to the python_examples directory for examples on how to use the iggy
library.
-## Running the Examples:
+## Running the Examples
Go to [python_examples/README.md](python_examples/README.md) for instructions
on running the examples.
-
## API Reference
For detailed documentation, visit [Apache Iggy's official
Docs](https://docs.iggy.rs/).
@@ -59,4 +58,4 @@ Apache iggy is distributed under the Apache 2.0 License. See
[LICENSE](LICENSE)
## Contact Information
-For questions, suggestions, or issues, contact the developers at [your email
address] or raise an issue on GitHub.
\ No newline at end of file
+For questions, suggestions, or issues, contact the developers at [your email
address] or raise an issue on GitHub.
diff --git a/foreign/python/python_examples/README.md
b/foreign/python/python_examples/README.md
index 53e49ac6..2fcdb59a 100644
--- a/foreign/python/python_examples/README.md
+++ b/foreign/python/python_examples/README.md
@@ -1,20 +1,20 @@
-## Python Examples
+# Python Examples
-### 1. Start the Producer:
+## 1. Start the Producer
Navigate to the `python_examples` directory and run:
-```
+```bash
python producer.py
```
-### 2. Start the Consumer:
+## 2. Start the Consumer
Still in the `python_examples` directory, run the consumer using a separate
terminal:
-```
+```bash
python consumer.py
```
Here's how the output should look (show images of the output):
-
\ No newline at end of file
+
diff --git a/justfile b/justfile
index 1e0b5207..a7d9f1ee 100644
--- a/justfile
+++ b/justfile
@@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.
-# Convinience commands for iggy
+# Convenience commands for iggy
# See https://github.com/casey/just
#
# Usage: just <command>
@@ -80,3 +80,6 @@ licenses-list-check:
licenses-list-fix:
./scripts/licenses-list.sh --update
+
+markdownlint:
+ markdownlint '**/*.md' --ignore-path .gitignore
diff --git a/scripts/licenses-list.sh b/scripts/licenses-list.sh
index 50c570b5..8739383d 100755
--- a/scripts/licenses-list.sh
+++ b/scripts/licenses-list.sh
@@ -78,7 +78,11 @@ cargo license --color never --do-not-bundle --all-features
>"$TEMP_FILE"
# Update mode
if [ "$MODE" = "update" ]; then
echo "Updating DEPENDENCIES.md..."
- cp "$TEMP_FILE" DEPENDENCIES.md
+ {
+ echo "# Dependencies"
+ echo ""
+ cat "$TEMP_FILE"
+ } >DEPENDENCIES.md
echo "DEPENDENCIES.md has been updated."
exit 0
fi
@@ -86,10 +90,19 @@ fi
# Check mode
if [ "$MODE" = "check" ]; then
echo "Checking if DEPENDENCIES.md is up to date..."
- if ! diff -q "$TEMP_FILE" DEPENDENCIES.md >/dev/null; then
+ # Create expected format for comparison
+ EXPECTED_FILE=$(mktemp)
+ trap 'rm -f "$TEMP_FILE" "$EXPECTED_FILE"' EXIT
+ {
+ echo "# Dependencies"
+ echo ""
+ cat "$TEMP_FILE"
+ } >"$EXPECTED_FILE"
+
+ if ! diff -q "$EXPECTED_FILE" DEPENDENCIES.md >/dev/null; then
echo "Error: DEPENDENCIES.md is out of date. Please run '$0 --update'
to update it."
echo "Diff:"
- diff -u DEPENDENCIES.md "$TEMP_FILE"
+ diff -u DEPENDENCIES.md "$EXPECTED_FILE"
exit 1
else
echo "DEPENDENCIES.md is up to date."
