When the World Wide Web was born, there was only one web server and one web client. The httpd web server was developed by the Centre d'Etudes et de Recherche Nucléaires (CERN) in Geneva, Switzerland. httpd has since become the generic name of the binary executable of many web servers. When CERN stopped funding the development of httpd, it was taken over by the Software Development Group of the National Center for Supercomputing Applications (NCSA). The NCSA also produced Mosaic, the first web browser, whose developers later went on to write the Netscape client.
Mosaic could fetch and view static documents and images served by the httpd server. This provided a far better means of disseminating information to large numbers of people than sending each person an email. However, the glut of online resources soon made search engines necessary, which meant that users needed to be able to submit data (such as a search string) and servers needed to process that data and return appropriate content.
A static document is one that exists in a constant state, such as a text file that doesn't change.
Search engines were first implemented by extending the web server, modifying its source code directly. Rewriting the source was not very practical, however, so the NCSA developed the Common Gateway Interface (CGI) specification. CGI became a standard for interfacing external applications with web servers and other information servers and generating dynamic information.
A CGI program can be written in virtually any language that can read from STDIN and write to STDOUT, regardless of whether it is interpreted (e.g., the Unix shell), compiled (e.g., C or C++), or a combination of both (e.g., Perl). The first CGI programs were written in C and needed to be compiled into binary executables. For this reason, the directory from which the compiled CGI programs were executed was named cgi-bin, and the source files directory was named cgi-src. Nowadays most servers come with a preconfigured directory for CGI programs called, as you have probably guessed, cgi-bin.