System Calls
When a computer is turned on, the program that gets executed first is called the ``operating system.'' It controls pretty much all activity in the computer. This includes who logs in, how disks are used, how memory is used, how the CPU is used, and how you talk with other computers. The operating system we use is called "Unix".
The way that programs talk to the operating system is via ``system calls.'' A system call looks like a procedure call (see below), but it's different -- it is a request to the operating system to perform some activity.
System calls are expensive. While a procedure call can usually be performed in a few machine instructions, a system call requires the computer to save its state, let the operating system take control of the CPU, have the operating system perform some function, have the operating system save its state, and then have the operating system give control of the CPU back to you. This concept is important, and will be seen time and time again in this class.
System calls are system dependent. Knowing this, it would not be a good idea to directly use system calls when portability cannot be neglected. System calls are also quite complex. Most often, it involves the duo of TRAP and RET (or some variations of those two). To implement system call, one would need specialized knowledge of I/O registers, the sequence of operations needed to use them; and most important of all, implement enough protection because I/O resources are generally shared among multiple users and/or processes.
System Calls for I/O
There are 5 basic system calls that Unix provides for file I/O.
1. int open(char *path, int flags [ , int mode ] );
2. int close(int fd);
3. int read(int fd, char *buf, int size);
4. int write(int fd, char *buf, int size);
5. off_t lseek(int fd, off_t offset, int whence);
You'll note that they look like regular procedure calls. This is how you program with them -- like regular procedure calls. However, you should know that they are different: A system call makes a request to the operating system. A procedure call just jumps to a procedure defined elsewhere in your program. That procedure call may itself make a system call (for example, fopen() calls open()), but it is a different call.
The reason the operating system controls I/O is for safety -- the computer must ensure that if my program has a bug in it, then it doesn't crash the system, and it doesn't mess up other people's programs that may be running at the same time or later. Thus, whenever you do disk or screen or network I/O, you must go through the operating system and use system calls.
These five system calls are defined fully in their man pages (do 'man -s 2 open', 'man -s 2 close', etc).
Open
Open makes a request to the operating system to use a file. The 'path' argument specifies what file you would like to use, and the 'flags' and 'mode' arguments specify how you would like to use it. If the operating system approves your request, it will return a ``file descriptor'' to you. This is a non-negative integer. If it returns -1, then you have been denied access, and you have to check the value of the variable "errno" to determine why. (That or use perror() -- see the lecture notes for Chapter 1).
All actions that you will perform on files will be done through the operating system. Whenever you want to do file I/O, you specify the file by its file descriptor. Thus, whenever you want to do file I/O on a specific file, you must first open that file to get a file descriptor.
Example: o1.c opens the file in1 for reading, and prints the value of the file descriptor. If you haven't copied over the file in1, then it will print -1, since in1 does not exist. If in1 does exist, then it will print 3, meaning that the open() request has been granted (i.e. a non-negative integer was returned). 3 is the file descriptor you obtained. Why 3? For each process, three I/O streams are opened by default, they are stdin, stdout and stderr. These three takes the file descriptors 0, 1 and 2.
There is a limit on how many files a program can open, because it takes resources to store all information needed to correctly handle an opened file. So, close all files you don't currently need to save some resources. As to the question of how many files can be opened, the answer varies from system to system and user to user. Some system allows as many as 2000 files, and sometimes a user's resource is limited. (Try running limit and find out). On the Sun machines we have, I found the limit to be 256 files, while a newer linux machine tells me that I can open 1024 files at the same time. Note: technically, the set of system constants that are assumed, i.e. the limits, can be found by tracing /usr/include/limits.h. However, great care must be taken since sometimes the necessary documentation is not available.
Note the value of 'flags' -- the man page for open() (or chapter 3 of the book) will give you a description of the flags and how they work. They are described in fcntl.h, which can be found in the directory /usr/include. (Note that fcntl.h merely includes /usr/include/sys/fcntl.h, so you'll have to look at that file to see what O_RDONLY and all really mean).
Example: o2.c tries to open the file "out1" for writing. That fails because out1 does not exist already. In order to open a new file for writing, you should open it with (O_WRONLY | O_CREAT | O_TRUNC) as the flags argument. See o3.c for an example of that. Notice that it creates the file out2, which is of zero length when the program terminates. Note also how o2.c and o3.c use perror() to flag errors.
UNIX> o2
o2: No such file or directory
UNIX> o3
UNIX> ls -l out*
-rw-r--r-- 1 plank 0 Sep 11 08:50 out2
UNIX>
Finally, the 'mode' argument should only be used if you are creating a new file. It specifies the protection mode of the new file. 0644 is the most typical value -- it says "I can read and write it; everyone else can only read it"
You can open the same file more than once. You will get a different fd each time. If you open the same file for writing more than once at a time, you may get bizarre results.
Close
Close() tells the operating system that you are done with a file descriptor. The OS can then reuse that file descriptor. The file c1.c shows some examples with opening and closing the file in1. You should look at it carefully, as it opens the file multiple times without closing it, which is perfectly legal in Unix.
Read
Read() tells the operating system to read "size" bytes from the file opened in file descriptor "fd", and to put those bytes into the location pointed to by "buf". It returns how many bytes were actually read. Consider the code in r1.c. When executed, you get the following:
UNIX> cat in1
Jim Plank
Claxton 221
UNIX> r1
called read(3, c, 10). returned that 10 bytes were read.
Those bytes are as follows: Jim Plank
called read(3, c, 99). returned that 12 bytes were read.
Those bytes are as follows: Claxton 221
UNIX>
There are a few things to note about this program. First, buf should point to valid memory. In r1.c, this is achieved by calloc()-ing space for c (read the man page on calloc() if you have not seen it before. Alternatively, I could have declared c to be a static array with 100 characters:
char c[100];
Second, I null terminate c after the read() calls to ensure that printf() will understand it.
Third, when read() returns 0, then the end of file has been reached. When you are reading from a file, if read() returns fewer bytes than you requested, then you have reached the end of the file as well. This is what happens in the second read() in r1.c.
Finally, note that the 10th character in the first read() call and the 12th character in the second are both newline characters. That is why you get two newlines in the printf() statement. One is in c, and the other is in the printf() statement.
Write
Write() is just like read(), only it writes the bytes instead of reading them. It returns the number of bytes actually written, which is almost invariably "size".
w1.c writes the string "cs360\n" to the file out3.
UNIX> w1
called write(3, "cs360\n", 6). it returned 6
UNIX> cat out3
cs360
UNIX>
Lseek
All open files have a "file pointer" associated with them. When the file is opened, the file pointer points to the beginning of the file. As the file is read or written, the file pointer moves. For example, in r1.c, after the first read, the file pointer points to the 11th byte in in1. You can move the file pointer manually with lseek(). The 'whence' variable of lseek specifies how the seek is to be done -- from the beginning of the file, from the current value of the pointer, and from the end of the file. The return value is the offset of the pointer after the lseek. Look at l1.c It does a bunch of seeks on the file in1. Trace it and make sure it all makes sense. How did I know to include sys/types.h and unistd.h? I typed "man -s 2 lseek".
Standard Input, Standard Output, and Standard Error
Now, every process in Unix starts out with three file descriptors predefined:
File descriptor 0 is standard input.
File descriptor 1 is standard output.
File descriptor 2 is standard error.
Thus, when you write a program, you can read from standard input, using read(0, ...), and write to standard output using write(1, ...).
Thus, we can write a very simple cat program (one that copies standard input to standard output) with one line: (this is in simpcat.c):
main()
{
char c;
while (read(0, &c, 1) == 1) write(1, &c, 1);
}
UNIX> simpcat < in1
Jim Plank
Claxton 221
UNIX>
We can have several opened file descriptors pointing to the same file. Suppose, we already have a file tfile on disk, that looks like the following:
UNIX> cat tfile
hope not
The following code, opens tfile twice, write once and read once via two different file descriptors.
#include < string.h >
#include < unistd.h >
#include < fcntl.h >
int main (void)
{
int fd[2];
char buf1[12] = "just a test";
char buf2[12];
fd[0] = open("tfile",O_RDWR);
fd[1] = open("tfile",O_RDWR);
write(fd[0],buf1,strlen(buf1));
write(1, buf2, read(fd[1],buf2,12));
close(fd[0]);
close(fd[1]);
return 0;
}
The output is then:
UNIX> a.out
just a testUNIX>
In this case, the content of tfile is consistent, regardless of using fd[0] or fd[1].
Now, take a look at the following code:
#include < stdio.h >
#include < stdlib.h >
#include < unistd.h >
int main (void)
{
int i, r, w;
char msg[12];
char buf[2] = {0, 0};
for (i = 0; i < 3; i ++) {
if ((r = read(i,buf,1))<0) {
sprintf(msg,"read f%d:%s",i,buf);
perror(msg);
}
if ((w = write(i,buf,1))<0) {
sprintf(msg,"write f%d:%s",i,buf);
perror(msg);
}
fprintf(stderr,"%d, r = %d, w = %d, char = %d\n",i,r,w,(int)(buf[0]));
}
return 0;
}
This piece of code is interesting because running it differently, we get very different results. In the following, bold letters signify keyboard inputs, while plain characters are the output. The file in1 is a plain text file:
UNIX> cat in1
ABCDEFGHIJK
UNIX> a.out |UNIX> a.out < in1 |UNIX> a.out < in1 > out1
A |write f0: Bad file descriptor |write f0: Bad file descriptor
A0, r = 1, w = 1, char = 65 |0, r = 1, w = -1, char = 65 |0, r = 1, w = -1, char = 65
|B |read f1: Bad file descriptor
1, r = 1, w = 1, char = 10 |B1, r = 1, w = 1, char = 66 |1, r = -1, w = 1, char = 65
C | |B
C2, r = 1, w = 1, char = 67 |2, r = 1, w = 1, char = 10 |B2, r = 1, w = 1, char = 66
Who can tell me what is going on?
Finally, let's take a peek at a concept called data driven programming. In the above program, outputting f0, f1, f2, etc. makes our life easy, since there is a consistent pattern. However, wouldn't outputing names like stdin, stdout and stderr be more clear? But then we cannot leverage a neat for loop any more. In fact, in many applications, when the number of possibilities for outputs goes way beyond 3, the code would look like some big mess like below:
....
for (i = 0; i < 3; i++) {
if (read(...) < 0) {
switch i {
case 0: sprintf(msg, "read stdin ...
case 1:
case 2:
....
}
perror....
}
if (write(...) < 0) {
switch i {
case 0: sprintf(msg, "write stdin ...
case 1:
case 2:
....
}
perror....
}
....
}
In light of this, take a look at the following instead:
#include < stdio.h >
#include < stdlib.h >
#include < unistd.h >
char msg[6][15] = {"read stdin",
"write stdin",
"read stdout",
"write stdout",
"read stderr",
"write stderr"};
int main (void)
{
int i, r, w;
char buf[2] = {0, 0};
for (i = 0; i < 3; i ++) {
if ((r = read(i,buf,1))<0)
perror(msg[i*2]);
if ((w = write(i,buf,1))<0)
perror(msg[i*2+1]);
fprintf(stderr,"%d, r = %d, w = %d, char = %d\n",i,r,w,(int)(buf[0]));
}
return 0;
}
The output is then:
UNIX> a.out < in1 > out1
write stdin: Bad file descriptor
0, r = 1, w = -1, char = 65
read stdout: Bad file descriptor
1, r = -1, w = 1, char = 65
B
B2, r = 1, w = 1, char = 66
UNIX>
We have already talked about distinguishing mechanism and policy. In this program, the output mechanism employed is simply a table lookup. The exact content of each output message should be a "policy" that gets defined separately. The actual code is this program is shortened. With this, we gain simplicity, robustness, and an ability to re-configure (by storing the output messages into a separate file). Hopefully, now you know that CS folks appreciate a very different kind of "simple and stupid", as opposed to how the general public interpret this term. Indeed, elegance in designs is what we strive for.
When a computer is turned on, the program that gets executed first is called the ``operating system.'' It controls pretty much all activity in the computer. This includes who logs in, how disks are used, how memory is used, how the CPU is used, and how you talk with other computers. The operating system we use is called "Unix".
The way that programs talk to the operating system is via ``system calls.'' A system call looks like a procedure call (see below), but it's different -- it is a request to the operating system to perform some activity.
System calls are expensive. While a procedure call can usually be performed in a few machine instructions, a system call requires the computer to save its state, let the operating system take control of the CPU, have the operating system perform some function, have the operating system save its state, and then have the operating system give control of the CPU back to you. This concept is important, and will be seen time and time again in this class.
System calls are system dependent. Knowing this, it would not be a good idea to directly use system calls when portability cannot be neglected. System calls are also quite complex. Most often, it involves the duo of TRAP and RET (or some variations of those two). To implement system call, one would need specialized knowledge of I/O registers, the sequence of operations needed to use them; and most important of all, implement enough protection because I/O resources are generally shared among multiple users and/or processes.
System Calls for I/O
There are 5 basic system calls that Unix provides for file I/O.
1. int open(char *path, int flags [ , int mode ] );
2. int close(int fd);
3. int read(int fd, char *buf, int size);
4. int write(int fd, char *buf, int size);
5. off_t lseek(int fd, off_t offset, int whence);
You'll note that they look like regular procedure calls. This is how you program with them -- like regular procedure calls. However, you should know that they are different: A system call makes a request to the operating system. A procedure call just jumps to a procedure defined elsewhere in your program. That procedure call may itself make a system call (for example, fopen() calls open()), but it is a different call.
The reason the operating system controls I/O is for safety -- the computer must ensure that if my program has a bug in it, then it doesn't crash the system, and it doesn't mess up other people's programs that may be running at the same time or later. Thus, whenever you do disk or screen or network I/O, you must go through the operating system and use system calls.
These five system calls are defined fully in their man pages (do 'man -s 2 open', 'man -s 2 close', etc).
Open
Open makes a request to the operating system to use a file. The 'path' argument specifies what file you would like to use, and the 'flags' and 'mode' arguments specify how you would like to use it. If the operating system approves your request, it will return a ``file descriptor'' to you. This is a non-negative integer. If it returns -1, then you have been denied access, and you have to check the value of the variable "errno" to determine why. (That or use perror() -- see the lecture notes for Chapter 1).
All actions that you will perform on files will be done through the operating system. Whenever you want to do file I/O, you specify the file by its file descriptor. Thus, whenever you want to do file I/O on a specific file, you must first open that file to get a file descriptor.
Example: o1.c opens the file in1 for reading, and prints the value of the file descriptor. If you haven't copied over the file in1, then it will print -1, since in1 does not exist. If in1 does exist, then it will print 3, meaning that the open() request has been granted (i.e. a non-negative integer was returned). 3 is the file descriptor you obtained. Why 3? For each process, three I/O streams are opened by default, they are stdin, stdout and stderr. These three takes the file descriptors 0, 1 and 2.
There is a limit on how many files a program can open, because it takes resources to store all information needed to correctly handle an opened file. So, close all files you don't currently need to save some resources. As to the question of how many files can be opened, the answer varies from system to system and user to user. Some system allows as many as 2000 files, and sometimes a user's resource is limited. (Try running limit and find out). On the Sun machines we have, I found the limit to be 256 files, while a newer linux machine tells me that I can open 1024 files at the same time. Note: technically, the set of system constants that are assumed, i.e. the limits, can be found by tracing /usr/include/limits.h. However, great care must be taken since sometimes the necessary documentation is not available.
Note the value of 'flags' -- the man page for open() (or chapter 3 of the book) will give you a description of the flags and how they work. They are described in fcntl.h, which can be found in the directory /usr/include. (Note that fcntl.h merely includes /usr/include/sys/fcntl.h, so you'll have to look at that file to see what O_RDONLY and all really mean).
Example: o2.c tries to open the file "out1" for writing. That fails because out1 does not exist already. In order to open a new file for writing, you should open it with (O_WRONLY | O_CREAT | O_TRUNC) as the flags argument. See o3.c for an example of that. Notice that it creates the file out2, which is of zero length when the program terminates. Note also how o2.c and o3.c use perror() to flag errors.
UNIX> o2
o2: No such file or directory
UNIX> o3
UNIX> ls -l out*
-rw-r--r-- 1 plank 0 Sep 11 08:50 out2
UNIX>
Finally, the 'mode' argument should only be used if you are creating a new file. It specifies the protection mode of the new file. 0644 is the most typical value -- it says "I can read and write it; everyone else can only read it"
You can open the same file more than once. You will get a different fd each time. If you open the same file for writing more than once at a time, you may get bizarre results.
Close
Close() tells the operating system that you are done with a file descriptor. The OS can then reuse that file descriptor. The file c1.c shows some examples with opening and closing the file in1. You should look at it carefully, as it opens the file multiple times without closing it, which is perfectly legal in Unix.
Read
Read() tells the operating system to read "size" bytes from the file opened in file descriptor "fd", and to put those bytes into the location pointed to by "buf". It returns how many bytes were actually read. Consider the code in r1.c. When executed, you get the following:
UNIX> cat in1
Jim Plank
Claxton 221
UNIX> r1
called read(3, c, 10). returned that 10 bytes were read.
Those bytes are as follows: Jim Plank
called read(3, c, 99). returned that 12 bytes were read.
Those bytes are as follows: Claxton 221
UNIX>
There are a few things to note about this program. First, buf should point to valid memory. In r1.c, this is achieved by calloc()-ing space for c (read the man page on calloc() if you have not seen it before. Alternatively, I could have declared c to be a static array with 100 characters:
char c[100];
Second, I null terminate c after the read() calls to ensure that printf() will understand it.
Third, when read() returns 0, then the end of file has been reached. When you are reading from a file, if read() returns fewer bytes than you requested, then you have reached the end of the file as well. This is what happens in the second read() in r1.c.
Finally, note that the 10th character in the first read() call and the 12th character in the second are both newline characters. That is why you get two newlines in the printf() statement. One is in c, and the other is in the printf() statement.
Write
Write() is just like read(), only it writes the bytes instead of reading them. It returns the number of bytes actually written, which is almost invariably "size".
w1.c writes the string "cs360\n" to the file out3.
UNIX> w1
called write(3, "cs360\n", 6). it returned 6
UNIX> cat out3
cs360
UNIX>
Lseek
All open files have a "file pointer" associated with them. When the file is opened, the file pointer points to the beginning of the file. As the file is read or written, the file pointer moves. For example, in r1.c, after the first read, the file pointer points to the 11th byte in in1. You can move the file pointer manually with lseek(). The 'whence' variable of lseek specifies how the seek is to be done -- from the beginning of the file, from the current value of the pointer, and from the end of the file. The return value is the offset of the pointer after the lseek. Look at l1.c It does a bunch of seeks on the file in1. Trace it and make sure it all makes sense. How did I know to include sys/types.h and unistd.h? I typed "man -s 2 lseek".
Standard Input, Standard Output, and Standard Error
Now, every process in Unix starts out with three file descriptors predefined:
File descriptor 0 is standard input.
File descriptor 1 is standard output.
File descriptor 2 is standard error.
Thus, when you write a program, you can read from standard input, using read(0, ...), and write to standard output using write(1, ...).
Thus, we can write a very simple cat program (one that copies standard input to standard output) with one line: (this is in simpcat.c):
main()
{
char c;
while (read(0, &c, 1) == 1) write(1, &c, 1);
}
UNIX> simpcat < in1
Jim Plank
Claxton 221
UNIX>
We can have several opened file descriptors pointing to the same file. Suppose, we already have a file tfile on disk, that looks like the following:
UNIX> cat tfile
hope not
The following code, opens tfile twice, write once and read once via two different file descriptors.
#include < string.h >
#include < unistd.h >
#include < fcntl.h >
int main (void)
{
int fd[2];
char buf1[12] = "just a test";
char buf2[12];
fd[0] = open("tfile",O_RDWR);
fd[1] = open("tfile",O_RDWR);
write(fd[0],buf1,strlen(buf1));
write(1, buf2, read(fd[1],buf2,12));
close(fd[0]);
close(fd[1]);
return 0;
}
The output is then:
UNIX> a.out
just a testUNIX>
In this case, the content of tfile is consistent, regardless of using fd[0] or fd[1].
Now, take a look at the following code:
#include < stdio.h >
#include < stdlib.h >
#include < unistd.h >
int main (void)
{
int i, r, w;
char msg[12];
char buf[2] = {0, 0};
for (i = 0; i < 3; i ++) {
if ((r = read(i,buf,1))<0) {
sprintf(msg,"read f%d:%s",i,buf);
perror(msg);
}
if ((w = write(i,buf,1))<0) {
sprintf(msg,"write f%d:%s",i,buf);
perror(msg);
}
fprintf(stderr,"%d, r = %d, w = %d, char = %d\n",i,r,w,(int)(buf[0]));
}
return 0;
}
This piece of code is interesting because running it differently, we get very different results. In the following, bold letters signify keyboard inputs, while plain characters are the output. The file in1 is a plain text file:
UNIX> cat in1
ABCDEFGHIJK
UNIX> a.out |UNIX> a.out < in1 |UNIX> a.out < in1 > out1
A |write f0: Bad file descriptor |write f0: Bad file descriptor
A0, r = 1, w = 1, char = 65 |0, r = 1, w = -1, char = 65 |0, r = 1, w = -1, char = 65
|B |read f1: Bad file descriptor
1, r = 1, w = 1, char = 10 |B1, r = 1, w = 1, char = 66 |1, r = -1, w = 1, char = 65
C | |B
C2, r = 1, w = 1, char = 67 |2, r = 1, w = 1, char = 10 |B2, r = 1, w = 1, char = 66
Who can tell me what is going on?
Finally, let's take a peek at a concept called data driven programming. In the above program, outputting f0, f1, f2, etc. makes our life easy, since there is a consistent pattern. However, wouldn't outputing names like stdin, stdout and stderr be more clear? But then we cannot leverage a neat for loop any more. In fact, in many applications, when the number of possibilities for outputs goes way beyond 3, the code would look like some big mess like below:
....
for (i = 0; i < 3; i++) {
if (read(...) < 0) {
switch i {
case 0: sprintf(msg, "read stdin ...
case 1:
case 2:
....
}
perror....
}
if (write(...) < 0) {
switch i {
case 0: sprintf(msg, "write stdin ...
case 1:
case 2:
....
}
perror....
}
....
}
In light of this, take a look at the following instead:
#include < stdio.h >
#include < stdlib.h >
#include < unistd.h >
char msg[6][15] = {"read stdin",
"write stdin",
"read stdout",
"write stdout",
"read stderr",
"write stderr"};
int main (void)
{
int i, r, w;
char buf[2] = {0, 0};
for (i = 0; i < 3; i ++) {
if ((r = read(i,buf,1))<0)
perror(msg[i*2]);
if ((w = write(i,buf,1))<0)
perror(msg[i*2+1]);
fprintf(stderr,"%d, r = %d, w = %d, char = %d\n",i,r,w,(int)(buf[0]));
}
return 0;
}
The output is then:
UNIX> a.out < in1 > out1
write stdin: Bad file descriptor
0, r = 1, w = -1, char = 65
read stdout: Bad file descriptor
1, r = -1, w = 1, char = 65
B
B2, r = 1, w = 1, char = 66
UNIX>
We have already talked about distinguishing mechanism and policy. In this program, the output mechanism employed is simply a table lookup. The exact content of each output message should be a "policy" that gets defined separately. The actual code is this program is shortened. With this, we gain simplicity, robustness, and an ability to re-configure (by storing the output messages into a separate file). Hopefully, now you know that CS folks appreciate a very different kind of "simple and stupid", as opposed to how the general public interpret this term. Indeed, elegance in designs is what we strive for.
No comments:
Post a Comment