pid and lock files

pid files are written by some programs to record their process ID while they are starting. This has multiple purposes:

• It's a signal to other processes and users of the system that that particular program is running, or at least started successfully.
• It allows one to write a script really easy to check if it's running and issue a plain killcommand if one wants to end it.
• It's a cheap way for a program to see if a previous running instance of it did not exit successfully.

Mere presence of a pid file doesn't guarantee that that particular process id is running, of course, so this method isn't 100% foolproof but "good enough" in a lot of instances. Checking if a particular PID exists in the process table isn't totally portable across UNIX-like operating systems unless you want to depend on the ps utility, which may not be desirable to call in all instances (and I believe some UNIX-like operating systems implement ps differently anyway).

Lock files are used by programs to ensure two (well-behaved) separate instances of a program, which may be running concurrently on one system, don't access something else at the same time. The idea is before the program accesses its resource, it checks for presence of a lock file, and if the lock file exists, either error out or wait for it to go away. When it doesn't exist, the program wanting to "acquire" the resource creates the file, and then other instances that might come across later will wait for this process to be done with it. Of course, this assumes the program "acquiring" the lock does in fact release it and doesn't forget to delete the lock file.

This works because the filesystem under all UNIX-like operating systems enforces serialization, which means only one change to the filesystem actually happens at any given time. Sort of like locks with databases and such.

Every process gets a process ID (pid) number. The system keeps track of processes by their pid. You can also control processes by their pid... kill 2134, or some-such.

Some programs use lock files to prevent other instances of the program running, or to have exclusive access to other files. yum and apt do, so that a second instance doesn't start up and the two mess up the system by both installing new packages at the same time.

A lock file isn't an actual lock on a file, it's just a file in a certain location that the programs are programmed to check. If file this_is_my_lock_file exists, do something. Any scripting language you use will be able to create and check for the existence of a file by name. Use of a lock file has to be written into a program. If program A creates a lock file on computer 2, but program B isn't programmed to check for the lock file, you might as well not have ever created it.

It might be better to put the scp-ing program on comp2... Program A on comp 2 checks for a lock file. If it finds one it exits. This keeps two instances from running. If it doesn't find a lock file, it creates one, then calls to comp1 for a new file. If there is no new file, it deletes the lock file then exits. If there is a new file, it copies it over to a temp file and closes the connection. It then creates a second lock file. This second lock file is the one program B should check for. With the second lock file in place, program A copies the temp file to where you want it, then deletes the temp file and both lock files, then exits.

By running program A on comp2, you have better control of the lock files. If program A was on comp1, and it creates a lock file on comp2, and the connection dies between them, then the lock file is still on comp2, and program B can't use the data file. If program A is on comp2 and the connection dies, program A can still access the lock file and delete it.

By using the temp file, if the connection dies and the whole file isn't copied over, you don't get stuck with half a data file on comp2. You might even want to have program A backup the file before replacing it with the temp file, just in case.

You might also want program B to create it's own lock file, so that program A won't try to replace the file while program B is currently using it. If program A finds program B's lock file, it can wait a short time and check again.