[Linux, OS] Alternative I/O Model(epoll) | Shared Library

2023.11.17 TIL

Code, Image source: The Linux Programming Interface, Michael Kerrisk

Alternative I/O Model (epoll)

• 지금까지 blocking 방법으로 처리
  • read() → sleep 걸림
  • 즉, 1개의 process가 한번에 1개의 fd로 I/O를 수행
  • 만약 read()에서 block이 걸리면?
• 해결책
  • Non-blocking + Polling
    • CPU가 계속 돌면서 확인 + 통신 → :(
  • Multi process/thread 이용
    • Data 도착 여부만 확인하고, child 혹은 다른 thread에서 data 통신 수행
  • 대체 I/O model
    • 하나의 process가 여러 fd로 I/O를 수행

tlpi-dist/altio/epoll_input.c

#include <sys/epoll.h>
#include <fcntl.h>
#include "tlpi_hdr.h"

#define MAX_BUF     1000        /* Maximum bytes fetched by a single read() */
#define MAX_EVENTS     5        /* Maximum number of events to be returned from
                                   a single epoll_wait() call */

int
main(int argc, char *argv[])
{
    int epfd, ready, fd, s, j, numOpenFds;
    struct epoll_event ev;
    struct epoll_event evlist[MAX_EVENTS];
    char buf[MAX_BUF];

    if (argc < 2 || strcmp(argv[1], "--help") == 0)
        usageErr("%s file...\n", argv[0]);

    epfd = epoll_create(argc - 1);
    if (epfd == -1)
        errExit("epoll_create");

    /* Open each file on command line, and add it to the "interest
       list" for the epoll instance */

    for (j = 1; j < argc; j++) {
        fd = open(argv[j], O_RDONLY);
        if (fd == -1)
            errExit("open");
        printf("Opened \"%s\" on fd %d\n", argv[j], fd);

        ev.events = EPOLLIN;            /* Only interested in input events */
        ev.data.fd = fd;
        if (epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev) == -1)
            errExit("epoll_ctl");
    }

    numOpenFds = argc - 1;

    while (numOpenFds > 0) {

        /* Fetch up to MAX_EVENTS items from the ready list of the
           epoll instance */

        printf("About to epoll_wait()\n");
        ready = epoll_wait(epfd, evlist, MAX_EVENTS, -1);
        if (ready == -1) {
            if (errno == EINTR)
                continue;               /* Restart if interrupted by signal */
            else
                errExit("epoll_wait");
        }

        printf("Ready: %d\n", ready);

        /* Deal with returned list of events */

        for (j = 0; j < ready; j++) {
            printf("  fd=%d; events: %s%s%s\n", evlist[j].data.fd,
                    (evlist[j].events & EPOLLIN)  ? "EPOLLIN "  : "",
                    (evlist[j].events & EPOLLHUP) ? "EPOLLHUP " : "",
                    (evlist[j].events & EPOLLERR) ? "EPOLLERR " : "");

            if (evlist[j].events & EPOLLIN) {
                s = read(evlist[j].data.fd, buf, MAX_BUF);
                if (s == -1)
                    errExit("read");
                printf("    read %d bytes: %.*s\n", s, s, buf);

            } else if (evlist[j].events & (EPOLLHUP | EPOLLERR)) {

                /* After the epoll_wait(), EPOLLIN and EPOLLHUP may both have
                   been set. But we'll only get here, and thus close the file
                   descriptor, if EPOLLIN was not set. This ensures that all
                   outstanding input (possibly more than MAX_BUF bytes) is
                   consumed (by further loop iterations) before the file
                   descriptor is closed. */

                printf("    closing fd %d\n", evlist[j].data.fd);
                if (close(evlist[j].data.fd) == -1)
                    errExit("close");
                numOpenFds--;
            }
        }
    }

    printf("All file descriptors closed; bye\n");
    exit(EXIT_SUCCESS);
}

• epfd = epoll_create(); → System call
• epoll_ctl() → (2개를 등록) → epoll_wait()에서 2개를 기다린다
• ready = epoll_wait(); → System call
  • 보통 read나 receive가 있어서 block이 됐는데 epoll은 nonblocking으로 wait을 한다
• s = read(); → 날아온 fd 정보에 맞게 event 정보를 가지고 실제 data read 수행

Shared Library

Static Libraries

• Object들의 복사본
• libname.a
• 생성 방법
  • ar options archive object-file
  • Options
    • r (replae), t(table of contents), d(delete)
    • cc -g -c mod1.c mod2.c mod3.c
    • ar -r libdemo.a mod1.o mod2.o mod3.o
    • rm mod1.o mod2.o mod3.o
    • ar tv libdemo.a
    • ar d libdemo.a mode3.o

• 사용
  • cc -g -c prog.c
  • cc -g -o prog prog.o libdemo.a
• 표준 directory(/usr/lib)에 있는 경우
  • cc -g -o prog prog.o -ldemo
• 표준 directory(/usr/lib)에 없 경우
  • cc -g -o prog prog.o -Lmylibdir -ldemo
    → 같이 linking을 해서 사용!

Shared Library

• 생성
  • gcc -g -c -fPIC -Wall mod1.c mod2.c mod3,c
  • gcc -g -shared -o libfoo.so mod1.o mod2.o mod3.o
  • gcc -g fPIC -Wall mod1.c mod2.c mod3.c -shared -o libfoo.so
    

• 사용
  • gcc -g -Wall -o prog prog.c libfoo.so
    • ./prog

      ./prog: error in loading shared libraries: libfoo.so: cannot open shared object file: No such file or directory

  • LD_LIBRARY_PATH
    • LD_LIBRARY_PATH=. ./prog
    • Called mod1-x1
    • Called mod2-x2

• 실행
  

• 유용한 Tool

  • ldd
    • 어떤 library를 쓰고있는지 경로와 함께 보여줌
    • ldd [filename]
      
  • The LD_LIBRARY_PATH environment variable

Shared Library Tools

• libdemo.so.1 → libdemo.so.1.0.2
• libdemo.so.2 → libdemo.so.2.0.0
• libreadline.so.5 → libreadline.so.5.0

동적으로 Library 적재

• dlopen() / dlsym() / dlclose() / dlerror()
• Plugin 개발에 사용
  • Plug-in file의 so file의 함수를 loading
• Vendor HAL 개발에 사용
  • Vendor HAL → SoC마다 다름
  • Framework Software는 공용

#include <dlfcn.h>

void *dlopen(const char *libfilename, int flags);  // Returns library handle on success, or NULL on error

#include <dlfcn.h>

const char *dlerror(void);  // Returns pointer to error-diagnostic string, or NULL if no error has occurred since previous call to dlerror()

#include <dlfcn.h>

int dlclose(void *handle);  // Returns 0 on success, or -1 on error

Symbol 주소 가져오기

#include <dlfcn.h>

int *dlsym(void *handle, char *symbol);  // Returns address on symbol, or NULL if symbol is not found

int *ip;

ip = (int *) dlsym(symbol, "myvar");
if (ip != NULL)
	printf("Value is %d\n", *ip);

funcp = dlsym(handle, symbol)

dynload

• ./dynload ./libdemo.so.1 x1
  Called mod1-x1
  
• LD_LIBRARY_PATH=. ./dynload ./libdemo.so.1 x1
  Called mod1-x1