Kbase P57906: How do Named Pipes work on UNIX ?
| Autor |
Progress Software Corporation - Progress |
| Acesso |
Público |
| Publicação |
04/03/2009 |
|
Status: Verified
GOAL:
How do Named Pipes work on UNIX ?
GOAL:
Using Named Pipes from within Progress on a UNIX platform
GOAL:
An in-depth explanation on Named Pipes on UNIX.
FACT(s) (Environment):
Progress 9.x
OpenEdge 10.x
UNIX
FIX:
This Solution provides an in-depth explanation on how Named Pipes work on UNIX, and what pitfalls should be taken into account when using them with Progress.
HOW NAMED PIPES WORK ON UNIX:
Named pipes are an inter-process communication (ipc) mechanism, and are created by the UNIX commands mknod or mkfifo. When using named pipes there are processes that write data to the pipe (writers) and that read data from it (readers), using the same system calls used for regular file I/O (open(), read(), write(), close() ...). The operating system provides synchronization between the processes accessing the named pipe depending on certain flags set by open(). By default this mechanism behaves as follows.
- A writer is can write data to the named pipe until the named pipe file buffer (FIFO queue) is full. The size of this buffer is determined by the PIPE_BUF system parameter in /usr/include/sys/param.h. (Default is usually 8192 bytes). If the buffer is full, the writer will block until a reader removes data from it.
- The writer can close the pipe at any time - there is no synchronization on closing this side of the pipe. If the writer attempts to write to the named pipe after the reader has closed it, the writer will receive a "Broken Pipe error (EPIPE).
- If no reader has the named pipe open, the writer will be blocked on open() until a reader opens the named pipe. The writer can reopen a pipe without blocking providing a reader still has the pipe open.
- A reader that has opened the named pipe and issued a read() system call will block until there is data in the buffer to read (written by a writer).
- The read() system call will read whatever data is in the queue up to the maximum specified in the read() call. For an unbuffered read this is one character, for buffered reads this is usually BUFSIZ (as specified in stdio.h).
- If the reader is blocked waiting for data and the writer closes the named pipe, the read() system call will return zero bytes and an EOF status.
- If the writer reopens the named pipe and writes to it, the reader can continue reading data until it closes the pipe on it's side.
- Data written to the pipe but not read will remain in the buffer provide that the named pipe is kept open by at least one processes. If the named pipe is closed by all readers and writers, any data not yet read from the it is lost and subsequent opens will start with an empty buffer.
The Progress 4GL will use the default behavior as described above.
USING THE NAMED PIPES FROM PROGRESS 4GL:
A 4GL reader will do the following:
- Open the pipe using INPUT [STREAM ... ] FROM <filename of pipe>.
- Read data using any statement that can read from a stream, for example: IMPORT STREAM ipipe UNFORMATTED myvar1.
- Close the pipe when reading finishes (which for continuous processes will be when the process is stopped).
The main thing to be aware of is that when the reader is blocked waiting for data and the writer closes the named pipe, the reader is returned an EOF status and per standard 4GL behavior closes the named pipe as well, which can result in data loss. A way to work around this would be to write a C shared library to read from the pipe and call that from the 4GL.
A 4GL writer will do the following:
- Open the pipe using OUTPUT [STREAM ... ] TO <filename of pipe> APPEND.
- Write data using any statement that can write to a stream, for example: PUT STREAM ipipe UNFORMATTED myvar1.
- Close the pipe when writing finishes.
When writing to a pipe, the following should be taken into account.
Unbuffered writing to a pipe means only one character is written at a time. This is slow when compared to buffered writes, and some applications may not be able to handle the unbuffered data properly.
However, if a buffered write is used, the buffer is not made available to the reader until the buffer is flushed. This flushing occur.s when more data is written to the buffer than the maximum buffer size (BUFSIZ set in stdio.h), or when the pipe is closed by the writer. In certain scenarios, this causes issues as the some readers will not always wait for the last buffer of a data segment to be flushed if it's not flushed immediately.
To ensure the last buffer is always flushed, either force a buffer overflow by passing the data with spaces or NULL characters, or close and reopen the pipe on the writer's side after each write operation - but ensure that the reader process can handle these scenarios properly. Older versions of Progress for example cannot, since the EOF that will be generated by the close on the writer side will trigger a close of the pipe on the reader's side as well, which may result in data loss..