참고 : https://strimzi.io/blog/2020/11/10/dynamically-changeable-logging/

Strimzi Kafka의 log

• log4j
  • Kafka brokers
  • Kafka Connect
  • Kafka MirrorMaker 2.0
• log4j2
  • Strimzi Kafka Bridge
  • Strimzi Topic Operator
  • Strimzi User Operator
  • Strimzi Cluster Operator

Configure log level

※ monitorInterval 는 수정하지 말기

• 수정하면 동적 재로드가 되지 않는다
• 롤링 업데이트 필요
• 수정 방법은 2가지

1. strimzi-cluster-operator Configmap의 log4j2.properties 수정
2. install/cluster-operator/050-ConfigMap-strimzi-cluster-operator.yaml 파일 수정

• 반영에 몇 초 소요

설정 확인

• broker

$ k exec my-cluster-kafka-0 -- bin/kafka-configs.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --entity-type broker-loggers --entity-name 0 --describe

• entity-name : kafka broker.id (pod명 뒤의 숫자)

Dynamic configs for broker-logger 0 are:
  com.yammer.metrics.reporting.JmxReporter=INFO sensitive=false synonyms={}
  io.netty.bootstrap.Bootstrap=INFO sensitive=false synonyms={}
  io.netty.buffer.AbstractByteBuf=INFO sensitive=false synonyms={}
  io.netty.buffer.ByteBufUtil=INFO sensitive=false synonyms={}
  io.netty.buffer.PoolThreadCache=INFO sensitive=false synonyms={}
  io.netty.buffer.PooledByteBufAllocator=INFO sensitive=false synonyms={}
  io.netty.channel.AbstractChannel=INFO sensitive=false synonyms={}
  io.netty.channel.AbstractChannelHandlerContext=INFO sensitive=false synonyms={}
  io.netty.channel.AbstractCoalescingBufferQueue=INFO sensitive=false synonyms={}
  io.netty.channel.ChannelHandlerMask=INFO sensitive=false synonyms={}
  io.netty.channel.ChannelInitializer=INFO sensitive=false synonyms={}
  io.netty.channel.ChannelOutboundBuffer=INFO sensitive=false synonyms={}
  io.netty.channel.DefaultChannelId=INFO sensitive=false synonyms={}
  io.netty.channel.DefaultChannelPipeline=INFO sensitive=false synonyms={}
  io.netty.channel.DefaultFileRegion=INFO sensitive=false synonyms={}
  io.netty.channel.DelegatingChannelPromiseNotifier=INFO sensitive=false synonyms={}
  io.netty.channel.MultithreadEventLoopGroup=INFO sensitive=false synonyms={}
  io.netty.channel.epoll.Native=INFO sensitive=false synonyms={}
  io.netty.channel.nio.AbstractNioChannel=INFO sensitive=false synonyms={}
  io.netty.channel.nio.NioEventLoop=INFO sensitive=false synonyms={}
  io.netty.channel.socket.nio.NioSocketChannel=INFO sensitive=false synonyms={}
  io.netty.channel.socket.nio.SelectorProviderUtil=INFO sensitive=false synonyms={}
  io.netty.handler.ssl.SslHandler=INFO sensitive=false synonyms={}
  io.netty.handler.ssl.SslUtils=INFO sensitive=false synonyms={}
  io.netty.resolver.AddressResolverGroup=INFO sensitive=false synonyms={}
  io.netty.util.NetUtil=INFO sensitive=false synonyms={}
  io.netty.util.NetUtilInitializations=INFO sensitive=false synonyms={}
  io.netty.util.Recycler=INFO sensitive=false synonyms={}
  io.netty.util.ReferenceCountUtil=INFO sensitive=false synonyms={}
  io.netty.util.ResourceLeakDetector=INFO sensitive=false synonyms={}
  io.netty.util.ResourceLeakDetectorFactory=INFO sensitive=false synonyms={}
  io.netty.util.concurrent.AbstractEventExecutor=INFO sensitive=false synonyms={}
  io.netty.util.concurrent.DefaultPromise=INFO sensitive=false synonyms={}
  io.netty.util.concurrent.DefaultPromise.rejectedExecution=INFO sensitive=false synonyms={}
  io.netty.util.concurrent.FastThreadLocalThread=INFO sensitive=false synonyms={}
  io.netty.util.concurrent.GlobalEventExecutor=INFO sensitive=false synonyms={}
  io.netty.util.concurrent.SingleThreadEventExecutor=INFO sensitive=false synonyms={}
  io.netty.util.internal.CleanerJava9=INFO sensitive=false synonyms={}
  io.netty.util.internal.InternalThreadLocalMap=INFO sensitive=false synonyms={}
  io.netty.util.internal.MacAddressUtil=INFO sensitive=false synonyms={}
  io.netty.util.internal.NativeLibraryLoader=INFO sensitive=false synonyms={}
  io.netty.util.internal.PlatformDependent=INFO sensitive=false synonyms={}
  io.netty.util.internal.PlatformDependent0=INFO sensitive=false synonyms={}
  io.netty.util.internal.SystemPropertyUtil=INFO sensitive=false synonyms={}
  io.netty.util.internal.logging.InternalLoggerFactory=INFO sensitive=false synonyms={}
  io.strimzi.kafka.agent.KafkaAgent=INFO sensitive=false synonyms={}
  kafka=INFO sensitive=false synonyms={}
  kafka.authorizer.logger=INFO sensitive=false synonyms={}
  kafka.cluster.Partition=INFO sensitive=false synonyms={}
  kafka.cluster.Replica=INFO sensitive=false synonyms={}
  kafka.common.ZkNodeChangeNotificationListener=INFO sensitive=false synonyms={}
  kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread=INFO sensitive=false synonyms={}
  kafka.controller=TRACE sensitive=false synonyms={}
  kafka.controller.ControllerChannelManager=TRACE sensitive=false synonyms={}
  kafka.controller.ControllerEventManager$ControllerEventThread=TRACE sensitive=false synonyms={}
  kafka.controller.KafkaController=TRACE sensitive=false synonyms={}
  kafka.controller.RequestSendThread=TRACE sensitive=false synonyms={}
  kafka.controller.TopicDeletionManager=TRACE sensitive=false synonyms={}
  kafka.controller.ZkPartitionStateMachine=TRACE sensitive=false synonyms={}
  kafka.controller.ZkReplicaStateMachine=TRACE sensitive=false synonyms={}
  kafka.coordinator.group.GroupCoordinator=INFO sensitive=false synonyms={}
  kafka.coordinator.group.GroupMetadata$=INFO sensitive=false synonyms={}
  kafka.coordinator.group.GroupMetadataManager=INFO sensitive=false synonyms={}
  kafka.coordinator.transaction.TransactionCoordinator=INFO sensitive=false synonyms={}
  kafka.coordinator.transaction.TransactionMarkerChannelManager=INFO sensitive=false synonyms={}
  kafka.log.LocalLog=INFO sensitive=false synonyms={}
  kafka.log.LogCleaner=INFO sensitive=false synonyms={}
  kafka.log.LogCleaner$CleanerThread=INFO sensitive=false synonyms={}
  kafka.log.LogCleanerManager$=INFO sensitive=false synonyms={}
  kafka.log.LogManager=INFO sensitive=false synonyms={}
  kafka.log.UnifiedLog=INFO sensitive=false synonyms={}
  kafka.log.UnifiedLog$=INFO sensitive=false synonyms={}
  kafka.network.ConnectionQuotas=INFO sensitive=false synonyms={}
  kafka.network.ControlPlaneAcceptor=INFO sensitive=false synonyms={}
  kafka.network.DataPlaneAcceptor=INFO sensitive=false synonyms={}
  kafka.network.Processor=FATAL sensitive=false synonyms={}
  kafka.network.RequestChannel$=WARN sensitive=false synonyms={}
  kafka.network.SocketServer=INFO sensitive=false synonyms={}
  kafka.request.logger=WARN sensitive=false synonyms={}
  kafka.server.AddPartitionsToTxnManager=INFO sensitive=false synonyms={}
  kafka.server.ClientQuotaManager$ThrottledChannelReaper=INFO sensitive=false synonyms={}
  kafka.server.ConfigAdminManager=INFO sensitive=false synonyms={}
  kafka.server.DefaultAlterPartitionManager=INFO sensitive=false synonyms={}
  kafka.server.DelayedFetch=INFO sensitive=false synonyms={}
  kafka.server.DelayedOperationPurgatory=INFO sensitive=false synonyms={}
  kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper=INFO sensitive=false synonyms={}
  kafka.server.DelayedProduce=INFO sensitive=false synonyms={}
  kafka.server.DynamicBrokerConfig=INFO sensitive=false synonyms={}
  kafka.server.FetchManager=INFO sensitive=false synonyms={}
  kafka.server.FetchSessionCache=INFO sensitive=false synonyms={}
  kafka.server.FinalizedFeatureChangeListener=INFO sensitive=false synonyms={}
  kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread=INFO sensitive=false synonyms={}
  kafka.server.FullFetchContext=INFO sensitive=false synonyms={}
  kafka.server.IncrementalFetchContext=INFO sensitive=false synonyms={}
  kafka.server.KafkaApis=FATAL sensitive=false synonyms={}
  kafka.server.KafkaConfig=INFO sensitive=false synonyms={}
  kafka.server.KafkaRequestHandler=INFO sensitive=false synonyms={}
  kafka.server.KafkaServer=INFO sensitive=false synonyms={}
  kafka.server.NodeToControllerRequestThread=INFO sensitive=false synonyms={}
  kafka.server.ReplicaFetcherManager=INFO sensitive=false synonyms={}
  kafka.server.ReplicaFetcherThread=INFO sensitive=false synonyms={}
  kafka.server.ReplicaFetcherTierStateMachine=INFO sensitive=false synonyms={}
  kafka.server.ReplicaManager=INFO sensitive=false synonyms={}
  kafka.server.ReplicaManager$LogDirFailureHandler=INFO sensitive=false synonyms={}
  kafka.server.TopicConfigHandler=INFO sensitive=false synonyms={}
  kafka.server.ZkAdminManager=INFO sensitive=false synonyms={}
  kafka.server.ZkConfigManager=INFO sensitive=false synonyms={}
  kafka.server.metadata.ZkMetadataCache=INFO sensitive=false synonyms={}
  kafka.utils.CoreUtils$=INFO sensitive=false synonyms={}
  kafka.utils.FileLock=INFO sensitive=false synonyms={}
  kafka.utils.Log4jControllerRegistration$=INFO sensitive=false synonyms={}
  kafka.zk.KafkaZkClient=INFO sensitive=false synonyms={}
  kafka.zookeeper.ZooKeeperClient=INFO sensitive=false synonyms={}
  org.apache.kafka=INFO sensitive=false synonyms={}
  org.apache.kafka.clients.ClientUtils=INFO sensitive=false synonyms={}
  org.apache.kafka.clients.ClusterConnectionStates=INFO sensitive=false synonyms={}
  org.apache.kafka.clients.CommonClientConfigs=INFO sensitive=false synonyms={}
  org.apache.kafka.clients.FetchSessionHandler=INFO sensitive=false synonyms={}
  org.apache.kafka.clients.NetworkClient=INFO sensitive=false synonyms={}
  org.apache.kafka.common.config.AbstractConfig=INFO sensitive=false synonyms={}
  org.apache.kafka.common.config.SecurityConfig=INFO sensitive=false synonyms={}
  org.apache.kafka.common.metrics.JmxReporter=INFO sensitive=false synonyms={}
  org.apache.kafka.common.metrics.Metrics=INFO sensitive=false synonyms={}
  org.apache.kafka.common.network.ChannelBuilders=INFO sensitive=false synonyms={}
  org.apache.kafka.common.network.NetworkReceive=INFO sensitive=false synonyms={}
  org.apache.kafka.common.network.PlaintextChannelBuilder=INFO sensitive=false synonyms={}
  org.apache.kafka.common.network.Selector=INFO sensitive=false synonyms={}
  org.apache.kafka.common.network.SslChannelBuilder=INFO sensitive=false synonyms={}
  org.apache.kafka.common.network.SslTransportLayer=INFO sensitive=false synonyms={}
  org.apache.kafka.common.protocol.Errors=INFO sensitive=false synonyms={}
  org.apache.kafka.common.record.MemoryRecords=INFO sensitive=false synonyms={}
  org.apache.kafka.common.record.MultiRecordsSend=INFO sensitive=false synonyms={}
  org.apache.kafka.common.requests.FetchMetadata=INFO sensitive=false synonyms={}
  org.apache.kafka.common.requests.OffsetFetchRequest=INFO sensitive=false synonyms={}
  org.apache.kafka.common.security.JaasUtils=INFO sensitive=false synonyms={}
  org.apache.kafka.common.security.ssl.DefaultSslEngineFactory=INFO sensitive=false synonyms={}
  org.apache.kafka.common.security.ssl.SslFactory=INFO sensitive=false synonyms={}
  org.apache.kafka.common.utils.AppInfoParser=INFO sensitive=false synonyms={}
  org.apache.kafka.common.utils.ConfigUtils=INFO sensitive=false synonyms={}
  org.apache.kafka.common.utils.KafkaThread=INFO sensitive=false synonyms={}
  org.apache.kafka.common.utils.LoggingSignalHandler=INFO sensitive=false synonyms={}
  org.apache.kafka.common.utils.Utils=INFO sensitive=false synonyms={}
  org.apache.kafka.controller=INFO sensitive=false synonyms={}
  org.apache.kafka.coordinator.group.assignor.UniformAssignor=INFO sensitive=false synonyms={}
  org.apache.kafka.metadata.properties.MetaPropertiesEnsemble=INFO sensitive=false synonyms={}
  org.apache.kafka.server.config.ConfigSynonym=INFO sensitive=false synonyms={}
  org.apache.kafka.server.util.KafkaScheduler=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.checkpoint.CleanShutdownFileHandler=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.epoch.LeaderEpochFileCache=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.log.AbstractIndex=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.log.LogConfig=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.log.LogDirFailureChannel=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.log.LogSegment=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.log.OffsetIndex=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.log.ProducerStateManager=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.log.SnapshotFile=INFO sensitive=false synonyms={}
  org.apache.kafka.storage.internals.log.TimeIndex=INFO sensitive=false synonyms={}
  org.apache.zookeeper=INFO sensitive=false synonyms={}
  org.apache.zookeeper.ClientCnxn=INFO sensitive=false synonyms={}
  org.apache.zookeeper.ClientCnxnSocket=INFO sensitive=false synonyms={}
  org.apache.zookeeper.ClientCnxnSocketNetty=INFO sensitive=false synonyms={}
  org.apache.zookeeper.CreateMode=INFO sensitive=false synonyms={}
  org.apache.zookeeper.ZKWatchManager=INFO sensitive=false synonyms={}
  org.apache.zookeeper.ZooKeeper=INFO sensitive=false synonyms={}
  org.apache.zookeeper.client.StaticHostProvider=INFO sensitive=false synonyms={}
  org.apache.zookeeper.common.NettyUtils=INFO sensitive=false synonyms={}
  org.apache.zookeeper.common.SSLContextAndOptions=INFO sensitive=false synonyms={}
  org.apache.zookeeper.common.X509Util=INFO sensitive=false synonyms={}
  org.apache.zookeeper.common.ZKConfig=INFO sensitive=false synonyms={}
  org.apache.zookeeper.common.ZKHostnameVerifier=INFO sensitive=false synonyms={}
  org.apache.zookeeper.common.ZKTrustManager=INFO sensitive=false synonyms={}
  org.apache.zookeeper.server.ZooKeeperThread=INFO sensitive=false synonyms={}
  org.eclipse.jetty.http.HttpFields=INFO sensitive=false synonyms={}
  org.eclipse.jetty.http.HttpGenerator=INFO sensitive=false synonyms={}
  org.eclipse.jetty.http.MimeTypes=INFO sensitive=false synonyms={}
  org.eclipse.jetty.http.PreEncodedHttpField=INFO sensitive=false synonyms={}
  org.eclipse.jetty.io.ArrayByteBufferPool=INFO sensitive=false synonyms={}
  org.eclipse.jetty.io.ManagedSelector=INFO sensitive=false synonyms={}
  org.eclipse.jetty.io.SelectorManager=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.AbstractConnector=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.AllowedResourceAliasChecker=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.HttpChannel=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.HttpConfiguration=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.SecureRequestCustomizer=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.Server=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.SymlinkAllowedResourceAliasChecker=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.handler.AbstractHandler=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.handler.AbstractHandlerContainer=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.handler.ContextHandler=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.handler.ContextHandler.v1_broker_state=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.handler.ContextHandler.v1_kraft_migration=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.handler.ContextHandler.v1_ready=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.handler.ContextHandlerCollection=INFO sensitive=false synonyms={}
  org.eclipse.jetty.server.handler.ErrorHandler=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.Jetty=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.URIUtil=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.component.AbstractLifeCycle=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.component.ContainerLifeCycle=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.log=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.resource.PathResource=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.resource.Resource=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.security.Credential=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.security.Password=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.ssl.SniX509ExtendedKeyManager=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.ssl.SslContextFactory=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.ssl.SslContextFactory.config=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.ssl.X509=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.thread.QueuedThreadPool=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.thread.ReservedThreadExecutor=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.thread.ShutdownThread=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.thread.ThreadPoolBudget=INFO sensitive=false synonyms={}
  org.eclipse.jetty.util.thread.strategy.EatWhatYouKill=INFO sensitive=false synonyms={}
  root=INFO sensitive=false synonyms={}
  state.change.logger=TRACE sensitive=false synonyms={}

• Connect

$ kubectl exec -ti my-connect-cluster-connect-0 -- curl http://localhost:8083/admin/loggers

{"org.apache.zookeeper":{"level":"ERROR","last_modified":null},
"org.reflections":{"level":"ERROR","last_modified":null},
"root":{"level":"INFO","last_modified":null}}

• MirrorMaker 2.0 (pod 없음)

$ kubectl exec -ti <kafka-mirror-maker-2-pod-name> -- curl http://localhost:8083/admin/loggers

• Strimzi Cluster Operator, Topic Operator, Entity Operator and Strimzi Kafka Bridge

/opt/strimzi/custom-config/log4j2.properties 경로에서 확인