kafka connector 등록 하면서
mydb 데이터베이스의 user 테이블을 감시하도록 설정했는데
이런 오류가 발생했다.
결론은 performance_schema에 정의되어있는 테이블 이름도 users 고
mydb에 정의되어있는 테이블 이름도 users 인데
users 테이블이 2개 있어서 나 일 안해~~
이거였다.
connector 정의 시 테이블 이름을 db.table 형태로 적어주니 해결되었다.
아래는 에러 내용
nrecoverable error while reading tables from the database (io.confluent.connect.jdbc.source.TableMonitorThread:224)
org.apache.kafka.connect.errors.ConnectException: The connector uses the unqualified table name as the topic name and has detected duplicate unqualified table names. This could lead to mixed data types in the topic and downstream processing errors. To prevent such processing errors, the JDBC Source connector fails to start when it detects duplicate table name configurations. Update the connector's 'table.whitelist' config to include exactly one table in each of the tables listed below.
[["mydb"."users", "performance_schema"."users"]]
at io.confluent.connect.jdbc.source.TableMonitorThread.tables(TableMonitorThread.java:152)
at io.confluent.connect.jdbc.JdbcSourceConnector.taskConfigs(JdbcSourceConnector.java:166)
at org.apache.kafka.connect.runtime.Worker.connectorTaskConfigs(Worker.java:382)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnector(DistributedHerder.java:2012)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnectorTasksWithExponentialBackoffRetries(DistributedHerder.java:1956)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnectorTasksWithRetry(DistributedHerder.java:1943)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$null$34(DistributedHerder.java:1888)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.runRequest(DistributedHerder.java:2156)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:467)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:368)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1589)
[2023-11-13 05:06:53,982] ERROR [my-source-connect|worker] WorkerConnector{id=my-source-connect} Connector raised an error (org.apache.kafka.connect.runtime.WorkerConnector:553)
org.apache.kafka.connect.errors.ConnectException: Encountered an unrecoverable error while reading tables from the database
at io.confluent.connect.jdbc.source.TableMonitorThread.fail(TableMonitorThread.java:226)
at io.confluent.connect.jdbc.source.TableMonitorThread.tables(TableMonitorThread.java:153)
at io.confluent.connect.jdbc.JdbcSourceConnector.taskConfigs(JdbcSourceConnector.java:166)
at org.apache.kafka.connect.runtime.Worker.connectorTaskConfigs(Worker.java:382)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnector(DistributedHerder.java:2012)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnectorTasksWithExponentialBackoffRetries(DistributedHerder.java:1956)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnectorTasksWithRetry(DistributedHerder.java:1943)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$null$34(DistributedHerder.java:1888)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.runRequest(DistributedHerder.java:2156)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:467)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:368)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1589)
Caused by: org.apache.kafka.connect.errors.ConnectException: The connector uses the unqualified table name as the topic name and has detected duplicate unqualified table names. This could lead to mixed data types in the topic and downstream processing errors. To prevent such processing errors, the JDBC Source connector fails to start when it detects duplicate table name configurations. Update the connector's 'table.whitelist' config to include exactly one table in each of the tables listed below.
[["mydb"."users", "performance_schema"."users"]]
at io.confluent.connect.jdbc.source.TableMonitorThread.tables(TableMonitorThread.java:152)
... 14 more
[2023-11-13 05:06:53,985] ERROR [Worker clientId=connect-1, groupId=connect-cluster] Failed to reconfigure connector's tasks (my-source-connect), retrying after backoff. (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1965)
org.apache.kafka.connect.errors.ConnectException: Encountered an unrecoverable error while reading tables from the database
at io.confluent.connect.jdbc.source.TableMonitorThread.fail(TableMonitorThread.java:226)
at io.confluent.connect.jdbc.source.TableMonitorThread.tables(TableMonitorThread.java:153)
at io.confluent.connect.jdbc.JdbcSourceConnector.taskConfigs(JdbcSourceConnector.java:166)
at org.apache.kafka.connect.runtime.Worker.connectorTaskConfigs(Worker.java:382)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnector(DistributedHerder.java:2012)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnectorTasksWithExponentialBackoffRetries(DistributedHerder.java:1956)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnectorTasksWithRetry(DistributedHerder.java:1943)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$null$34(DistributedHerder.java:1888)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.runRequest(DistributedHerder.java:2156)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:467)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:368)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1589)
Caused by: org.apache.kafka.connect.errors.ConnectException: The connector uses the unqualified table name as the topic name and has detected duplicate unqualified table names. This could lead to mixed data types in the topic and downstream processing errors. To prevent such processing errors, the JDBC Source connector fails to start when it detects duplicate table name configurations. Update the connector's 'table.whitelist' config to include exactly one table in each of the tables listed below.
[["mydb"."users", "performance_schema"."users"]]
at io.confluent.connect.jdbc.source.TableMonitorThread.tables(TableMonitorThread.java:152)
... 14 more
[2023-11-13 05:06:53,989] INFO [Worker clientId=connect-1, groupId=connect-cluster] Skipping reconfiguration of connector my-source-connect since it is not running (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1996)
[2023-11-13 05:06:54,246] INFO [Worker clientId=connect-1, groupId=connect-cluster] Skipping reconfiguration of connector my-source-connect since it is not running (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1996)
[2023-11-13 05:06:57,042] INFO 127.0.0.1 - - [12/11월/2023:20:06:57 +0000] "GET /connectors HTTP/1.1" 200 21 "-" "Apache-HttpClient/4.5.14 (Java/17.0.6)" 3 (org.apache.kafka.connect.runtime.rest.RestServer:62)
[2023-11-13 05:06:59,945] INFO 127.0.0.1 - - [12/11월/2023:20:06:59 +0000] "GET /connectors/my-source-connect/status HTTP/1.1" 200 2571 "-" "Apache-HttpClient/4.5.14 (Java/17.0.6)" 2 (org.apache.kafka.connect.runtime.rest.RestServer:62)
Hello velog!