Streaming Connectors

Flink 内建连接器


Flink 提供了一系列常见的内建连接器,用于支持各类系统的 DataSources 数据导入、Sinks 数据导出。

以下是 Apache RabbitMQ Connectors 的使用示例:

/* RabbitMQ DataSources 使用 */

final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// 设置 checkpoint 担保
env.enableCheckpointing(...);

final RMQConnectionConfig connectionConfig = new RMQConnectionConfig.Builder()
    .setHost("localhost")
    .setPort(5000)
    ...
    .build();
    
final DataStream<String> stream = env
    .addSource(new RMQSource<String>(
        connectionConfig,            // 配置 RabbitMQ 连接
        "queueName",                 // 连接的 RabbitMQ 队列名称
        true,                        // use correlation ids; can be false if only at-least-once is required
        new SimpleStringSchema()))   // 序列化,message 消息转为 Java Object
    .setParallelism(1);              // non-parallel source is only required for exactly-once

/* RabbitMQ Sinks 使用 */

final DataStream<String> stream = ...
final RMQConnectionConfig connectionConfig = new RMQConnectionConfig.Builder()
    .setHost("localhost")
    .setPort(5000)
    ...
    .build();
stream.addSink(new RMQSink<String>(
    connectionConfig,            // config for the RabbitMQ connection
    "queueName",                 // name of the RabbitMQ queue to send messages to
    new SimpleStringSchema()));  // serialization schema to turn Java objects to messages