August 2007

Mom vs. MSN

My Mom likes MSN.com. It's been her homepage for years, and I've accepted the fact that I won't be able to get her to switch to a better portal. However, every time Mom gets a new computer, she somehow gets tricked into signing up for MSN dial-up service for $9.99/month, despite the fact that she already has a broadband Internet connection.

Mom's a smart woman, so I'm not sure why this keeps happening. I think the new computers' desktops are always cluttered with crapware icons, and as she clicks them all to see what they do, one of them offers to help her "get connected to the Internet," even though she's already connected to the Internet. Then an animated translucent butterfly pesters her until the clicks "OK."

I'm sure there are people who actually would find it helpful for a new Windows machine to assist them in setting up a dial-up account. But, it would be even more helpful if the MSN take-your-money wizard would detect whether a dial-up connection is really needed.

Next time I help set up a new computer, I'll be sure to delete anything that says "MSN" before turning it over to the owner.

Python Server Start, Take 2

A couple of months ago, I posted Python Server Start, a simple template for starting implementation of a network server in Python. I got a comment from "dt" suggesting that what I really wanted to use was the standard Python SocketServer module.

Today, I had to write a "real server" in Python, so I finally got around to looking into SocketServer. The documentation wasn't helpful, but the source code for the module was straightforward, so I figured things out pretty quickly.

After work, I decided to create a more generic version of what I'd done while I was on the clock. What follows is my new "starting point" for implementing a server process in Python. It's about 300 lines long, which is a bit large for a "Hello, world!" kind of program, but it has these nifty new features:

  • The server starts a daemon process, disconnected from the user's terminal, like it should.
  • The server writes to a log file
  • It implements a simple protocol between client and server. Basically, the client just sends its command-line arguments to the server, and the server processes the command and sends output back, which the client writes to standard output. (This protocol should, of course, be replaced with whatever protocol your real server has to handle; the template is just in place for testing and demonstration.)
  • It can work with TCP/IP sockets, or can use UNIX domain sockets (on platforms that support them).

I've only tried it on Mac OS X and Linux. It will need some work for Windows, but thankfully, I haven't had to do much Windows programming lately, so I'm not going to worry about it.

Making the necessary changes to use a base of ForkingTCPServer, ThreadingTCPServer, ThreadingUnixStreamServer, or other variations is left as an exercise for the reader.

I welcome any suggestions for improvement.

#!/usr/bin/env python

"""Server start

This is a template for a Python-based server daemon derived from
SocketServer.  Hack it up as needed.

This script implements both the server daemon and a command-line
client that can issue requests against it.  The template client-server
protocol is very simple: the client simply sends the command-line
arguments to the server, and the server returns output which the
client writes to its standard output.  Change the protocol as needed
for your purposes.

The template contains a few UNIXisms.  Modification may be needed for
a Windows-based server.

References:
- Source for ServerSocket.py (standard Python module)
- Source for BaseHTTPServer.py (standard Python module)
- http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/278731

"""

version = '1.0'

usage = """usage: %prog [options] command [arg...]

commands:
  start     start the server daemon
  stop      stop the server daemon
  status    return server daemon status
  echo      server echoes arguments
  add A B   return A+B

Example session:
  %prog start     # starts daemon
  %prog status    # print daemon's status
  %prog add 15 8  # prints "15 + 8 = 23"
  %prog stop      # stops daemon"""


import SocketServer
import optparse
import os
import os.path
import resource
import socket
import sys
import tempfile
import time

# We can use either a TCPServer or a UnixStreamServer (assuming the OS
# supports UNIX domain sockets).  We just need to define the
# appropriate ServerBase class and then customize a few things based
# upon which base we're using.

#ServerBase = SocketServer.TCPServer
ServerBase = SocketServer.UnixStreamServer
if ServerBase == SocketServer.TCPServer:
    # TODO: replace with appropriate port number
    server_address = ('', 54545)
elif ServerBase == SocketServer.UnixStreamServer:
    # TODO: replace with appropriate socket file path
    server_address = os.path.join(tempfile.gettempdir(), 'server_socket')

# Path to log file
# TODO: Change to appropriate path and name
server_log = os.path.join(tempfile.gettempdir(), 'server.log')


class RequestHandler(SocketServer.StreamRequestHandler):

    """Request handler

    An instance of this class is created for each connection made
    by a client.  The Server class invokes the instance's
    setup(), handle(), and finish() methods.

    The template implementation here simply reads a single line from
    the client, breaks that up into whitespace-delimited words, and
    then uses the first word as the name of a "command."  If there is
    a method called "do_COMMAND", where COMMAND matches the
    commmand name, then that method is invoked.  Otherwise, an error
    message is returned to the client.

    """

    def handle(self):
        """Service a newly connected client.

        The socket can be accessed as 'self.connection'.  'self.rfile'
        can be used to read from the socket using a file interface,
        and 'self.wfile' can be used to write to the socket using a
        file interface.

        When this method returns, the connection will be closed.
        """
        
        # Read a single request from the input stream and process it.
        # TODO: Change as needed for actual client-server protocol.
        request = self.rfile.readline()
        if request:
            self.server.log('request %s: %s',
                            self.connection.getpeername(), request.rstrip())
            try:
                self.process_request(request)
            except Exception, e:
                self.server.log('exception: %s' % str(e))
                self.wfile.write('Error: %s\n' % str(e))
        else:
            self.server.log('error: unable to read request')
            self.wfile.write('Error: unable to read request')


    def process_request(self, request):
        """Process a request.

        This method is called by self.handle() for each request it
        reads from the input stream.

        This implementation simply breaks the request string into
        words, and searches for a method named 'do_COMMAND',
        where COMMAND is the first word.  If found, that method is
        invoked and remaining words are passed as arguments.
        Otherwise, an error is returned to the client.
        """

        words = request.split()
        if len(words) == 0:
            self.server.log('error: empty request')
            self.wfile.write('Error: empty request\n')
            return

        command = words[0]
        args = words[1:]

        methodname = 'do_' + command
        if not hasattr(self, methodname):
            self.server.log('error: invalid command')
            self.wfile.write('Error: "%s" is not a valid command\n' % command)
            return
        method = getattr(self, methodname)
        method(*args)


    def do_stop(self, *args):
        """Process a 'stop' command"""
        self.wfile.write('Stopping server\n')
        self.server.stop()


    def do_echo(self, *args):
        """Process an 'echo' command"""
        self.wfile.write(' '.join(args) + '\n')


    def do_status(self, *args):
        """Process a 'status' command"""
        self.wfile.write('Server Version:    %s\n' % version)
        self.wfile.write('Process ID:        %d\n' % os.getpid())
        self.wfile.write('Parent Process ID: %d\n' % os.getppid())
        self.wfile.write('Server Socket:     %s\n' % str(server_address))
        self.wfile.write('Server Log:        %s\n' % server_log)


    def do_add(self, a, b):
        """Process an 'add' command"""
        answer = int(a) + int(b)
        self.wfile.write('%s + %s = %s\n' % (a, b, answer))


class Server(ServerBase):

    """Server implementation

    """

    def __init__(self, server_address):
        """Constructor"""
        self.__daemonize()

        if ServerBase == SocketServer.UnixStreamServer:
            # Delete the socket file if it already exists
            if os.access(server_address, 0):
                os.remove(server_address)

        ServerBase.__init__(self, server_address, RequestHandler)


    def log(self, format, *args):
        """Write a message to the server log file"""
        try:
            message = format % args
            timestamp = time.strftime('%Y-%m-%d %H:%M:%S')
            f = open(server_log, 'a+')
            f.write('%s %s\n' % (timestamp, message))
            f.close()
        except Exception, e:
            print str(e)


    def serve_until_stopped(self):
        """Serve requests until self.stop() is called.

        This is an alternative to BaseServer.serve_forever()
        """

        self.log('started')
        self.__stopped = False
        while not self.__stopped:
            self.handle_request()
        self.log('stopped')


    def stop(self):
        """Stop handling requests.

        Calling this causes the server to drop out of
        serve_until_stopped().
        """

        self.__stopped = True


    def __daemonize(self):
        """Create daemon process.

        Based upon recipe provided at
        http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/278731
        """

        UMASK = 0
        WORKDIR = '/'
        MAXFD = 1024
        if hasattr(os, 'devnull'):
            REDIRECT_TO = os.devnull
        else:
            REDIRECT_TO = '/dev/null'

        try :
            if os.fork() != 0:
                os._exit(0)

            os.setsid()

            if os.fork() != 0:
                os._exit(0)

            os.chdir(WORKDIR)
            os.umask(UMASK)
        except OSError, e:
            self.log('exception: %s %s', e.strerror, e.errno)
            raise Exception, "%s [%d]" % (e.strerror, e.errno)
        except Exception, e:
            self.log('exception: %s', str(e))
        
        maxfd = resource.getrlimit(resource.RLIMIT_NOFILE)[1]
        if maxfd == resource.RLIM_INFINITY:
            maxfd = MAXFD
        for fd in range(0, maxfd):
            try:
                os.close(fd)
            except OSError:
                pass

        os.open(REDIRECT_TO, os.O_RDWR)
        os.dup2(0, 1)
        os.dup2(0, 2)


def run_server(options, args):
    """Run a server daemon in the current process."""
    svr = Server(server_address)
    svr.serve_until_stopped()
    svr.server_close()


def do_request(options, args):
    """Send request to the server and process response."""
    if ServerBase == SocketServer.UnixStreamServer:
        s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    elif ServerBase == SocketServer.TCPServer:
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # Send request
    # TODO: Change as needed for actual client-server protocol
    s.connect(server_address)
    s.sendall(' '.join(args) + '\n')

    # Print response
    # TODO: Change as needed for actual client-server protocol
    sfile = s.makefile('rb')
    line = sfile.readline()
    while line:
        print line,
        line = sfile.readline()


#
# MAIN
#
if __name__ == '__main__':
    optparser = optparse.OptionParser(usage=usage,
                                      version=version)
    (options, args) = optparser.parse_args()

    if len(args) == 0:
        optparser.print_help()
        sys.exit(-1)

    if args[0] == 'start':
        run_server(options, args[1:])
    else:
        do_request(options, args)

Screwed E-Mail

I have my own domain that I've been using for e-mail for the past few years. Most of my e-mail goes there, and then gets forwarded to a Yahoo! mail account, which in turn gets downloaded to my Macbook. This gives me a couple of nice features: I can read everything in the nice Mac Mail app when I'm at home, but I can also access everything via the web when I'm elsewhere.

Another nice feature was that I had the mail server configured such that anything@mydomain.com would get to me. This made it easy to essentially create new e-mail accounts whenever I needed a new one. For example, if Microsoft wants me to register for something, I'd give them "microsoft@mydomain.com" as my e-mail address. Since every website in the world wants my e-mail address, I figured that doing this would give me a way to create a different e-mail address for everybody who needs to contact me, making it easy to filter out stuff that I didn't want.

This worked great, until today. Today, my hosting provider, A2 Hosting, decided to disable the feature that automatically forwards everything to one place. Now, I need to create accounts or forwarders for every single address that I'd like to handle.

The problem is that I've been doing this for a long time, and I don't have a list of all the addresses I've used. Every business and every website I've interacted with in the past couple of years will no longer be able to contact me, unless I can remember them. So, now I'm not going to be getting a lot of e-mails that are important to me. I'm screwed.

The provider disabled this feature due to spam. It is too easy for spammers to just randomly generate e-mail addresses for every domain, and there is a cost to the provider for every e-mail they forward. I do sympathize, but it doesn't change the fact that I am now screwed by their policy change.

So, I think I need to find another hosting provider, so I can start getting e-mail again. Anybody out there happy with theirs?

Alternatively, I suppose I could write a script to find all the e-mail addresses for which I've received mail for the past couple of years. Switching providers sounds a lot simpler.