IO Systems and Operations

Reminder: A View of OS Services

os에서 여러 시스템 기능을 제공하고 있다.
application이 system call interface를 통해 아래 서비스들을 사용할 수 있다.

I/O (Input/Output)

• The main job of a computer is I/O
  • Its another (major) job, Computing or Processing, is merely secondary
• The role of OS in computer I/O is to control devices and to manage I/O operations
  컴퓨터 I/O에서 OS의 역할은 장치를 제어하고 I/O 작업을 관리하는 것입니다

The Role of OS: I/O Device Control

• I/O devices vary so widely in their function and speed
  -> Varied methods are needed to control them
  • These methods form the “I/O subsystem of the OS kernel”, which separates the rest of the kernel from the complexities of managing I/O devices
    이 method들은 "OS 커널의 I/O 서브시스템"을 형성하는데,(즉, method들이 서브시스템에 속한다) 이는 커널의 나머지 부분과 I/O 디바이스들을 관리하는 복잡한 것들을 분리합니다.(계층적 구조를 가진다)
• The basic I/O hardware elements, such as ports, buses, and device controllers, accommodate a wide variety of I/O devices
  • Port: A connection point between a device and a computer (e.g., a serial port)
  • Bus: A common set of wires, shared by devices, and a rigidly defined protocol that specifies a set of messages that can be sent on the wires(데이터 전달하는 wire이며 bus protocol 존재)

device driver는 device controller를 제어하는 일부이다. 둘은 다르다

여기서부터는 소프트웨어 파트이다.

• To encapsulate the details and oddities(특이성) of different devices, the OS kernel is structured to use device-driver modules
  • Device Driver: A uniform device-access interface to the I/O subsystem, much as system calls provide a standard interface between the application and the OS
    I/O 서브시스템에 대한 균일한 장치 접근 인터페이스(system call이 애플리케이션과 OS 간의 표준 인터페이스를 제공하는 것처럼)

I/O subsystem이 device driver에서 제공하는 interface 함수를 통해서 device controller와 통신하여 I/O device를 제어할 수 있다

커널과 하드웨어가 통신할 수 있다.

A Typical PC Bus Structure

하드웨어 관점에서 I/O device들을 나타내고 있다.

중간에 PCI bus를 통해서 i/o device와 메모리 사이에서 data가 이동한다.
여러 device controller(ex. graphics controller)에 필요한 I/O device(ex. monitor)가 존재함을 확인할 수 있다.

cache는 processor 내부에 존재한다. 당연한 이야기인데 까먹지마렴.

A Kernel I/O Structure

kernel I/O subsystem이 device driver를 통해서 device controller와 통신 가능하다.

The Role of OS: I/O Operation Management

• I/O operations or transactions occur when a program receives bytes from a source (e.g., a keyboard, mouse, file, and sensor) or sends bytes to a destination (e.g., a display, file, printer, and actuator)
  -> Most modern OS, including Unix/Linux, use the “stream model” to support I/O operations

Stream Model

• Any input or output transaction can be viewed as a flow of bytes (=stream) from a source to a destination
• The operating system creates and manages streams based upon the function calls made by the program
  • An I/O stream connection can be established using the fopen() function, and broken using the fclose() function

The Steps in Creating, Using, and Closing a Stream

• When a stream is created, one can think of it as having an address from which bytes are sent and an address at which bytes are received.
  • Each address is at a memory location controlled by the OS

Transporting Bytes on Streams

• There are several functions that can be used to transport bytes on streams:
  • These include the familiar fprintf() and fscanf() functions, which are specialized to work only with ASCII formatted text data
    여기에는 친숙한 fprintf() 및 fscanf() 함수가 포함되며, 이 함수는 ASCII 형식의 텍스트 데이터에서만 작동하도록 특수화되어 있습니다
  • The generic functions for this purpose are fread() and fwrite()
    -> At the heart of both fprintf() and fscanf() functions are calls to fread() and fwrite() to actually move the bytes after the desired byte manipulations have been completed
    fprintf()와 fscanf() 함수의 핵심은 fread()와 fwrite() 함수 모두 원하는 바이트 조작이 완료된 후 실제로 바이트를 이동하기 위한 호출입니다
  • There are a number of other functions as well, such as fgetc(), fputc(), gets(), and puts()

fprintf()

• Bytes are being sent down a stream using the fprintf() function
  

fscanf()

• Bytes are being transported on a stream using the fscanf() function

• fscanf() and fprintf() functions perform byte manipulations besides the byte transfers.
  fscanf () 및 fprintf () 함수는 바이트 전송 이외에 바이트 조작을 수행합니다.
  
  fscanf() 자리에 %s가 아니라 %d가 들어갔으니까 정수형으로 받아지지..

fread() & fwrite()

• Synopsis
  • size_t fread(void ptr, size_t size, size_t nmemb, FILE stream);
  • size_t fwrite(const void ptr, size_t size, size_t nmemb, FILE stream);
• Description
  • The function fread() reads nmemb elements of data, each size bytes long, from the stream pointed to by stream, storing them at the location given by ptr.
    함수 fread()는 스트림이 가리키는 스트림으로부터 각각의 크기 바이트 길이의 데이터의 nmemb 요소를 판독하여 ptr이 지정한 위치에 저장합니다.
  • The function fwrite() writes nmemb elements of data, each size bytes long, to the stream pointed to by stream, obtaining them from the location given by ptr.
• Return value
  • On success, fread() and fwrite() return the number of items read or written
    -> An indicator of the number of bytes actually moved along the stream
  • If an error occurs, the return value is a short item count (or zero).

fread()

• Bytes are read from a source into the program regardless of what the bytes represent.

fread(text,sizeof(char), 15,fpt)도 가능하다
ptr 에 저장할 배열의 원소 하나의 크기를 입력한다.
(ex: char a[3] : sizeof(char), struct str_t[3] : sizeof(struct str_t))

fopen 할때 모드 혹시 모르니 유의 하기!!

fwrite()

• Bytes are moved from the program into a destination
  • The parameters of the fwrite() are identical to those in the fread() function
    
  • Executing this code will produce a file data2.txt including “Fortytwo 42 …”
  • Note that this example uses the strlen() function to determine the number of cells in the array to send on the stream, which in this case is 40

Short Question…

• What would you expect here?

실제로 읽어들인 40이 출력된다.

System I/O Functions

• Another set of functions that can be used for I/O
  • These functions include open(), close(), read(), and write()
  • They look very similar to the f-versions of the I/O functions, i.e., fopen(), fclose(), fread(), and fwrite() functions, which are standardized in the C library and therefore expected to behave similarly regardless of the underlying operating system
• The non-f I/O functions are “system calls”
  시스템 콜을 위한 referencing이다. 내부를 열어보면 system call 이외에 아무것도 없음
  • Depending on which OS they are called from, they may behave differently.
  • A general rule of thumbs is that the non-f versions are NOT buffered, while the f-versions are buffered

Standard Streams

• Most programs get user input from a keyboard and display text output to a screen. Therefore, every time a program is started, the OS automatically creates three streams:
  • Standard in (stdin): It connects keyboard to program
  • Standard out (stdout): It connects program to display
  • Standard error (stderr)
    • It connects program to a secondary display that is intended only for displaying errors
    • stderr stream is intended as a backup output stream for programs
• The standard streams are most commonly used by calling the scanf() and printf() functions
  • While fscanf() can receive bytes from any stream, scanf() is “hardwired” to the stdin stream. The same is true with regards to printf() and fprintf()
    fscanf()는 모든 스트림에서 바이트를 수신할 수 있지만 scanf()는 stdin 스트림에 "하드와이어"됩니다. printf() 및 fprintf()에 대해서도 마찬가지입니다

Example

• In the following example, the pair of scanf() & fscanf() and printf() & fprintf() function calls perform the exact same operation;

• The definitions for the standard streams can be found in the include file stdio.h

Buffering

• A buffer is a temporary storage or the “middlemen” between sender and receiver of bytes on a stream (i.e., between two devices or between a device and an application)
  • An additional piece of memory that is used to moderate(조정하다) the flow of bytes from the source to the destination
• Buffering is done for several reasons:
  • To cope with device speed mismatch
    • What if the sender puts bytes into the stream faster than the receiver can handle?
    • What if a program is in the middle of a calculation and is not prepared to receive any bytes?
  • To cope with device data-transfer size mismatch
    - Common in computer networking, where buffers are used widely for fragmentation and reassembly of messages
    버퍼가 메시지의 조각화 및 재조립을 위해 널리 사용되는 컴퓨터 네트워킹에서 일반적입니다

Three Types of Buffering

• Based on how the temporary storage is “flushed(비우다)”
  • Block buffering
    • The buffer is flushed when it receives a certain amount of data (e.g. 1KB, 16KB, 64KB, etc…)
    • Commonly used for large data transfers, such as file I/O
  • Line buffering
    • The buffer is flushed when it receives a newline character(‘\n’)
    • Typically used for text-based I/O, such as when interacting with a user
  • Unbuffered
    - The buffer doesn’t act as a buffer and is flushed every bytes
    - Buffering may be completely turned off when responsiveness is critical
    응답성이 중요한 경우 버퍼링이 완전히 해제될 수 있음
• The type of buffering on an existing stream can be changed by the setvbuf() or setbuf() function

Example code

[RESULT]
i=0 i=1 i=2 i=3 i=4

프로그램이 종료될때 buffer가 flush 된다.

[RESULT]
i=0
i=1
i=2
i=3
i=4
stdout has a line buffer

[RESULT]
i=0 i=1 i=2 i=3 i=4

The buffer can be forced to flush using fflush()

Pipes

• The term “PIPE”
  • Used in contexts where streams are reconnected to alternate sources or destinations
  • The process of connecting and reconnecting streams is referred to as piping, or pipelining
  • An example of piping is to reconnect the standard out stream so that it simultaneously sends the same bytes to a file and a display
  • pipe() and dup() are the file I/O functions (i.e., system calls) that can be used to manipulate stream connections
    pipe()와 dup()은 스트림 연결을 조작하는 데 사용할 수 있는 파일 I/O 함수(system call)입니다
    

Piping symbols

• 3 pipelining symbols
  • OS automatically creates three streams for every running program.
  • Most shells provide the ability to redirect those streams at startup

Piping Example (1/3)

Piping Example (2/3)

Piping Example (3/3)

• Connect the stdout stream of one program to the stdin stream of a second program
  

Pipe chaining

• Pipeline chaining
  • Multiple piping redirections can be done simultaneously
• Example
  • Both the stdin and stdout of the summer are being redirected
  • bingo also has its stdout stream redirected to a file

Files

• What is it?
  • Contiguous logical address space
  • An image, movie, or database can be stored as a one dimensional array of bytes by writing all the elements out in a long list

File Pointer

• A marker used to keep track of the location for reading or writing on a stream.
• When opening a file, the file pointer points to the 1st byte in the file.
  Each time a byte is read, the file pointer automatically advances to the next byte.
  • When reading multiple bytes, the file pointer advances beyond all the read bytes
• Functions for manipulating the file pointer:
  • fseek(): moves it to a new value without reading or writing any data on the stream
  • ftell(): obtain its current value

Example

• Consider the following data in a file: abcdef
  

fseek()

• Synopsis
  • int fseek(FILE *stream, long offset, int whence);
• Description
  • The fseek() function sets the file position indicator for the stream pointed to by stream
  • The new position, measured in bytes, is obtained by adding offset bytes to the position specified by whence
  • If whence is set to SEEK_SET, SEEK_CUR, or SEEK_END, the offset is relative to the start of the file, the current position indicator, or end-of-file, respectively
• Return value
  • Upon successful completion, fseek() returns 0
  • If an error occurs, the return value is -1

Example: fileseek.c

fread에서 읽은 크기에 따라 fseek에서 그 크기만큼 되돌아가야한다는 로직 기억하고.. 지금은 1개 읽었으니 -1이고, 4개 읽으면 -4로 가야한다.
각 매개변수가 무엇이 들어가는지 시험전 반드시 확인.. 나올 가능성 높다

File Attributes

• They are metadata associated with computer files
  • Name, size, date, time, and file system permissions,

• Permission indicates who is allowed to access the file, and what types of access are allowed
  • r: read, w:write, x:excute, -: permission denied
  • On a UNIX system, it can be changed using the “chmod” system program

-rwxr-xr-x의 예제를 살펴보자
먼저 가장 앞의 -는 파일의 유형이다 지금은 일반적인 파일이고
그 다음부터 3자리씩 owner, group, others의 권한을 의미한다.

stat()

• Synopsis
  • int stat(const char pathname, struct stat buf);
• Description
  • It returns information about a file, in the buffer pointed to by buf
  • No permissions are required on the file itself, but—in the case of stat()- execute (search) permission is required on all of the directories in pathname that lead to the file
    stat은 파일로 이어지는 경로 이름의 모든 디렉토리에 대해 실행(검색) 권한을 가지고 있다.
• Return value
  • On success, zero(0) is returned
  • If an error occurs, the return value is -1

stat data structure

stat example code

빨간색 2진수는 permisson을 나타낸 것이니 시험에도 나올수 있으니 유의!!

별로 어렵진 않음..

Directory

• Directory
  • A directory is an organizational tool for files
  • List of filenames and auxiliary information for each file
  • OS manages directories in much the same manner as it manages files
• These tables can be accessed using the opendir(), readdir(), and closedir() functions
  

Example of directory code

Devices

• One of the neat aspects of UNIX is that any peripheral, device, or piece of hardware connected to the computing system can be accessed as if it was a file
  UNIX의 깔끔한 측면 중 하나는 컴퓨팅 시스템에 연결된 모든 주변 장치, 장치 또는 하드웨어를 파일처럼 액세스할 수 있다는 점입니다
• This concept is often referred to as “file-based I/O”
  • An I/O transaction with a device is handled similarly to an I/O transaction with a file. They both make use of streams and buffers, and use the same I/O functions

Device Driver

• A device driver is a set of functions used to access a device
  • open(), close(), read(), write(), lseek(), dup(), …
• The functions associated with a particular device are executed when the device is accessed through its device filename
  특정 장치와 관련된 함수는 장치 파일 이름을 통해 장치에 액세스할 때 실행됩니다
• A device driver layer within a kernel I/O structure
  • Making the kernel I/O subsystem independent of the hardware
    -> simplify the job of the OS developer

• Most operating systems have an escape (or back door) that transparently passes arbitrary commands from an application to a device driver
  대부분의 운영 체제에는 애플리케이션에서 디바이스 드라이버로 임의의 명령을 투명하게 전달하는 탈출구(또는 백도어)가 있습니다
  • In UNIX, this system call is ioctl(), which enables an application to access any functionality that can be implemented by any device driver, without the need to invent a new system call

ioctl()

• Synopsis
  • #include <sys/ioctl.h>
    int ioctl(int fd, unsigned long request, …);
• Description
  • The ioctl() is used for an application to communicate with I/O device drivers
  • The 1st argument fd must be an open file descriptor (= a device identifier that connects the application to the driver)
  • The 2nd argument is a device-dependent request code (= an integer that selects one of the commands implemented in the driver)
  • determined by device driver developers
  • The 3rd argument is a pointer to an arbitrary data structure in memory that enables the application and driver to communicate any necessary control information or data

Major/minor number

• The device identifier in UNIX/Linux is a tuple of “major and minor” device numbers
  • Major number is the device type, and it is used by the OS to route I/O requests to the appropriate device driver.
    OS에서 I/O 요청을 적절한 장치 드라이버로 라우팅하는 데 사용됩니다
  • Minor number is the instance of that device, and it is used to provide a unique ID for multiple device filenames that use the same device driver functions

The Overall Structure of Device-Driver System

• Linux splits all devices into three classes:
  • Block devices
  • Character devices
  • Network devices
    

Block and Character Devices

• Block device
  • Include a disk driver
  • Transfer a block of bytes as a unit
  • Expected to understand commands such as read(), write(), seek()
• Character-stream device
  • Include a keyboard, mouse, serial port, printer, audio board
  • Transfer bytes one by one
  • Commands include get(), put() (printf, scanf도 가능)
  • Libraries layered on top allow line-at-a-time access, with buffering and editing services (e.g., when a user types a backspace, the preceding character is removed from the input stream)
    상단에 계층화된 라이브러리는 버퍼링 및 편집 서비스를 통해 한 번에 한 줄씩(라인 단위로) 처리할 수 있습니다
    (예: 사용자가 백스페이스를 입력하면 입력 스트림에서 이전 문자가 제거됨)

Network Devices

• Because the performance and addressing characteristics of network I/O differ significantly from those of disk I/O, most OS provide a network I/O interface that is different from the read()-write()-seek() interface used for disks
  • One interface available in many OSs, including UNIX/Linux and Windows, is the network socket interface
• Just like a wall socket for electricity (-> any electrical appliance can be plugged in), the system calls in the socket interface enable an application to
  • create a socket,
  • connect a local socket to a remote address,
  • listen for any remote application to plug into the local socket,
  • send and receive packets over the connection

Summary

• Streams
  • Any input or output transaction can be viewed as a flow of bytes
• Buffers
  • It is a temporary storage between the sender and receiver
• Pipes
  • The process of connecting and reconnecting streams
• Files
  • Files stored as a one-dimensional array of bytes
• Devices
  • Devices can be accessed as though it were a file