If I'm writing C-language CGI programs, how much stuff is Apache doing
that I lose control of? It's nice to have it doing the
listen-on-port-80 part, but I don't want it deciding to twiddle
response headers etc for me. Basically I'm looking to get down as low
as possible and still be able to run on a shared server without
sacrificing too much in the portability area. Is CGI the most
efficient way to work within the Apache framework?

--
Legacy browsers never heard of emerging standards.
(contact via http://www.ren-prod-inc.com/hug_soft)

hug <contact_info@sig_line.clickit> wrote:

>If I'm writing C-language CGI programs, how much stuff is Apache doing
>that I lose control of? It's nice to have it doing the
>listen-on-port-80 part, but I don't want it deciding to twiddle
>response headers etc for me. Basically I'm looking to get down as low
>as possible and still be able to run on a shared server without
>sacrificing too much in the portability area. Is CGI the most
>efficient way to work within the Apache framework?

I forgot to mention the nph part. I'm under the impression that
nph-cgi indicates to Apache that it must leave the response headers
alone.

I ***ume that Apache is still doing its logging thing, and the other
config/htaccess stuff, nyet? How much of that can generally be
avoided in a shared server configuration?

This may belong in some other group: On Unix/Linux systems is there a
mechanism for running a program that will remain memory-resident, or
does one need to run it as a cron job that uses a tcp port interface
and point Apache at a stub that talks to it?

--
Legacy browsers never heard of emerging standards.
(contact via http://www.ren-prod-inc.com/hug_soft)

hug wrote:

> If I'm writing C-language CGI programs, how much stuff is Apache doing
> that I lose control of? It's nice to have it doing the
> listen-on-port-80 part, but I don't want it deciding to twiddle
> response headers etc for me.

It follows the CGI 1.1 standard -- basically, it sets certain environment
variables which your program can read, feeds POST data into your program's
STDIN, sends your program's STDOUT back to the client, and logs your
program's STDERR to Apache's error_log. The CGI standard is freely
available -- Google it.

> Is CGI the most efficient way to work within the Apache framework?

No, probably an Apache module would be, but that wouldn't be suitable for
shared hosting.

> I ***ume that Apache is still doing its logging thing, and the other
> config/htaccess stuff, nyet? How much of that can generally be avoided
> in a shared server configuration?

Yes -- with CGI, Apache will continue to process .htaccess files, perform
logging and so forth.

> This may belong in some other group: On Unix/Linux systems is there a
> mechanism for running a program that will remain memory-resident,

Any program that doesn't crash or exit will (obviously) remain resident in
memory. If what you really mean is that you want a way of forcing a
program to run in the *background* you have two main ways of doing this:

1. Within the program itself: it calls the system fork() function to fork
itself in half. The "original" half will receive a return value >= 1 from
the fork() process, and can now exit. The "new" half will receive a return
value of 0 and should continue running, say, listening on a TCP port.

2. When calling the program, append an ampersand (!!!) to the system call,
which will spawn the program in the background, and instantly return
control to the original process.

> or does one need to run it as a cron job that uses a tcp port interface
> and point Apache at a stub that talks to it?

Running a program from cron that listens on a TCP Port is certainly
possible, but would be highly unusual -- it wouldn't do what I think you
want to do.

What you seem to be aiming at is an efficient process for running
server-side CGI-like programs. My recommendation would be to use
existing mechanisms that have been programmed by very clever people.
Jumping through hoops for efficiency's sake is not worth it: Moore's
Law[1] will optimise your program far faster than you ever could.

In particular, you mention loading something into memory and then a
"stub" connecting to it for each request. ***uming your reason for this
is that the program holds a lot of data which is time-consuming to read
into memory from disk for every request, then consider using a SQL
database -- the database daemon will continue to run in the background
between requests, negating the need to continuously load and unload
data from the disk. The database can hold not just data, but also
indexes, views and functions. Again, do not worry about speed or
reliability: the database has been written by very clever people.

____
1. http://foldoc.org/?Moore's+Law

--
Toby A Inkster BSc (Hons) ARCS
Contact Me ~ http://tobyinkster.co.uk/contact

Toby Inkster <EMAIL REMOVED> wrote:

>hug wrote:
>
>> If I'm writing C-language CGI programs, how much stuff is Apache doing
>> that I lose control of? It's nice to have it doing the
>> listen-on-port-80 part, but I don't want it deciding to twiddle
>> response headers etc for me.
>
>It follows the CGI 1.1 standard -- basically, it sets certain environment
>variables which your program can read, feeds POST data into your program's
>STDIN, sends your program's STDOUT back to the client, and logs your
>program's STDERR to Apache's error_log. The CGI standard is freely
>available -- Google it.

I spent some time reading the CGI 1.1 standard this morning before
posting the question; it answers the detailed questions about
implementation, but still left me slightly confused about the context.

It talked about the HTTPD which I realize will in most cases be
Apache, but exactly what the HTTPD would be doing on top of what my
code would be doing wasn't quite clear... what I ***ume is: listening
to port-80, screening based on .htaccess, figuring out what cgi module
to invoke, logging, and tweaking headers before sending the data back
to the client if parsed headers are allowed. But there could be other
stuff I'm unaware of, and I prefer surprises only on birthdays etc.

When you say it logs STDERR to Apache's error log, I ***ume that means
in-adddition-to the normal Apache log records rather than instead-of?

>> Is CGI the most efficient way to work within the Apache framework?
>
>No, probably an Apache module would be, but that wouldn't be suitable for
>shared hosting.

I definitely want to remain able to run on a shared server.

>> I ***ume that Apache is still doing its logging thing, and the other
>> config/htaccess stuff, nyet? How much of that can generally be avoided
>> in a shared server configuration?
>
>Yes -- with CGI, Apache will continue to process .htaccess files, perform
>logging and so forth.
>
>> This may belong in some other group: On Unix/Linux systems is there a
>> mechanism for running a program that will remain memory-resident,
>
>Any program that doesn't crash or exit will (obviously) remain resident in
>memory. If what you really mean is that you want a way of forcing a
>program to run in the *background* you have two main ways of doing this:

I'm not clear on the *nix difference between background and
foreground. If a program has been invoked and has not crashed/exited,
and is listening on some TCP port, it's available to provide a service
isn't it? Does it matter whether it wears a "background" or
"foreground" hat?

>1. Within the program itself: it calls the system fork() function to fork
>itself in half. The "original" half will receive a return value >= 1 from
>the fork() process, and can now exit. The "new" half will receive a return
>value of 0 and should continue running, say, listening on a TCP port.

If the original half is going to exit and leave the fork running,
what's the benefit of that over just doing the work in the original
half and not exiting?

>2. When calling the program, append an ampersand (!!!)

Coded as &amp; no doubt. <g>

> to the system call,
>which will spawn the program in the background, and instantly return
>control to the original process.
>
>> or does one need to run it as a cron job that uses a tcp port interface
>> and point Apache at a stub that talks to it?
>
>Running a program from cron that listens on a TCP Port is certainly
>possible, but would be highly unusual -- it wouldn't do what I think you
>want to do.

I don't know what constitutes the unusual on a *nix server; I'm not
sure why it wouldn't do what I want it to do though. cron is only a
way of kicking it off so it can be initialized before (with luck) the
first request hits it. I'm not a *nix guy so there are plenty of
better-ways to do things in that environment that I don't know about
yet.

>What you seem to be aiming at is an efficient process for running
>server-side CGI-like programs. My recommendation would be to use
>existing mechanisms that have been programmed by very clever people.
>Jumping through hoops for efficiency's sake is not worth it: Moore's
>Law[1] will optimise your program far faster than you ever could.

I do prefer to reinvent only the parts of the wheel that don't do what
needs to be done.

>In particular, you mention loading something into memory and then a
>"stub" connecting to it for each request. ***uming your reason for this
>is that the program holds a lot of data which is time-consuming to read
>into memory from disk for every request,

Yes, it not only needs to load a lot of data that can be preprocessed
for efficiency, it also needs to handle information that can be
accessed and modified in response to multiple httpd requests, and keep
that central information in-sync and stored on disk. I expect that
the mechanism I'm looking for would amount to a *nix "daemon", but I
need to implement it in a way that allows it to run on a shared server
without systemwide configuration changes.

> then consider using a SQL
>database -- the database daemon will continue to run in the background
>between requests, negating the need to continuously load and unload
>data from the disk. The database can hold not just data, but also
>indexes, views and functions.

I was on the project that developed what I believe was the first
commercially available SQL database, it was called IBM Database 2, and
was released sometime around 1983. I didn't care much for SQL then,
and nothing I've seen since has made it any more appealing.

> Again, do not worry about speed or
>reliability: the database has been written by very clever people.

[A rant about "very clever people" and the pointy-haired idiots that
direct them to do Very Stupid Things has been deleted here.]

The "daemon" approach sounds like something else that would kick my
code out of the shared-server realm.

--
Legacy browsers never heard of emerging standards.
(contact via http://www.ren-prod-inc.com/hug_soft)

Fleeing from the madness of the . jungle
hug <contact_info@sig_line.clickit> stumbled into news:alt.www.webmaster
and said:

> ...
> I was on the project that developed what I believe was the first
> commercially available SQL database, it was called IBM Database 2, and
> was released sometime around 1983.

Logica had their project unwrapped in the 70s iirc - no idea which
actually broke outta the shrinkwrap first.

--
William T***o

http://williamt***o.com/words/what-is-usenet.asp

"William T***o" <EMAIL REMOVED> wrote:

>Fleeing from the madness of the . jungle
>hug <contact_info@sig_line.clickit> stumbled into news:alt.www.webmaster
>and said:
>
>> ...
>> I was on the project that developed what I believe was the first
>> commercially available SQL database, it was called IBM Database 2, and
>> was released sometime around 1983.
>
>Logica had their project unwrapped in the 70s iirc - no idea which
>actually broke outta the shrinkwrap first.

I'm not familiar with Logica, but it doesn't much matter to me who
invented SQL, whether it originated as System/R at IBM Almaden
Research, or as Hoohah3 in Bangladesh -- I still think SQL sucks; not
everything is a table or a nail, but SQL is the popular hammer and I
just shrug and walk away.

--
Legacy browsers never heard of emerging standards.
(contact via http://www.ren-prod-inc.com/hug_soft)