Topic
kafka topic은 cluster안에 있는 데이터 스트림을 뜻함
데이터베이스 테이블과 유사, 하지만 제약이 없음( 원하는 모든 것을 Topic에 전송 가능, 데이터 검증이 없기 때문)
A topic is identified by its name
모든 종류의 메세지 포맷을 지원
The sequence of messages is called a data stream
Partitions and offsets
Topics are split in partitions (example : 100 partitions)
-
Messages within each partition are ordered
-
Each message within a partition gets an incremental id, called offset
Kafka topics are immutable : once data is written to a partition, it cannot be changed. ( 삭제, 수정 불가)
Topics, partitions and offset - important notes
Once the data is written to a partition, it cannot be changed ( immutability )
Data is kept only for a limited time ( default is one week - configurable )
Offset only have a meaning for a specific partition.
Offsets are not re-used even if previous messages have been deleted
Order is guranteed only within a partition ( not across partitions )
Data is assigned randomly to a partition unless a key is provided
You can have as many partitions per topic as you want
