Skip to content

Split client API into flume-client artifact - #511

Open
ppkarwasz wants to merge 3 commits into
feat/rename-artifactsfrom
feat/flume-client
Open

ppkarwasz wants to merge 3 commits into
feat/rename-artifactsfrom
feat/flume-client

Conversation

@ppkarwasz

Copy link
Copy Markdown
Member

This PR splits classes used by the RPC client into its own flume-client artifact. This will allow Flume users (like log4j-flume) to only depend on flume-client.

At the same time this partially solves the split org.apache.flume package problem and defines the roles of the various artifacts:

  1. flume-client is a lightweight API to connect to Flume, which only needs a runtime (flume-avro-client or flume-thrift-client) to work.
  2. flume-sdk will be a required dependency for Flume plugin implementations (sources, sinks, channels).
  3. Only Flume agents will need flume-core.

Client applications only need `Event`, the event builders and `RpcClient`, but had to depend on the SDK, which also carries the plugin API. Splitting them also removes a split package: `Event` moves to `org.apache.flume.event`, so the client artifact owns `org.apache.flume.api`, `org.apache.flume.event` and `org.apache.flume.util`.

`RpcClient` now extends `Closeable` and reports failures with `IOException` and the new `RpcDeliveryException`, so the client no longer needs the plugin-side `EventDeliveryException` and `FlumeException`. The event readers move to `org.apache.flume.serialization` and `EventHelper` to `org.apache.flume.core`, so that `org.apache.flume.event` lives in one artifact.

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
`flume-auth` never used the SDK, so dropping that dependency lets a Thrift client depend on it without pulling in the plugin API. The Thrift client, source and sink move to role-first packages in the rpc repository.

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
The spooling directory constants and `PlatformDetect` move out of the SDK to the new `flume-spooldir-source` artifact, which is their only user; this also removes two packages split between the SDK and the core. The aliases and launchers point to the new Avro and spooling directory packages, which also fixes the `avro` deserializer and `avro_event` serializer aliases that referred to non-existent classes.

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants