Saturday, December 22, 2012

Communication between Nginx upload module and Tornado, Python

Basic communication flow:

User (Internet, Port 80)===upload===>Nginx (listens 80) saves files===redirect modified POST headers via port 8080===> Python Tornado  (listens to 8080)====> return users information

1. Setting up Nginx

- I put the upload form (index.html) on /var/www/html/php, so I set this as the working directory (bad naming, but nevermind!)

==========
Index.html (from the Official example)


<html>
<head>
<title>Test upload</title>
</head>
<body>
<h2>Select files to upload</h2>
<form name="upload" method="POST" enctype="multipart/form-data" action="/upload">
<input type="file" name="file1"><br>
<input type="file" name="file2"><br>
<input type="file" name="file3"><br>
<input type="file" name="file4"><br>
<input type="file" name="file5"><br>
<input type="file" name="file6"><br>
<input type="submit" name="submit" value="Upload">
<input type="hidden" name="test" value="value">
</form>
</body>
</html>



Nginx.conf:



worker_processes  20;

error_log  /var/log/nginx/error.log debug;

working_directory /var/www/html/php;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    server {
        listen       80;
        client_max_body_size 100m;

        # Upload form should be submitted to this location
        location / {
        root /var/www/html/php;
        index index.html index.htm;
        }
        location /upload {
            # Pass altered request body to this location
            upload_pass   @test;

            # Store files to this directory
            # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist << You must manually create these subdir before you upload, because Nginx upload does not support creating new directories

            upload_store /tmp 1;
         
            # Allow uploaded files to be read only by user
            upload_store_access user:r;

            # Set specified fields in request body
            upload_set_form_field "${upload_field_name}_name" $upload_file_name;
            upload_set_form_field "${upload_field_name}_content_type" $upload_content_type;
            upload_set_form_field "${upload_field_name}_path" $upload_tmp_path;

            # Inform backend about hash and size of a file
            upload_aggregate_form_field "${upload_field_name}_md5" $upload_file_md5;
            upload_aggregate_form_field "${upload_field_name}_size" $upload_file_size;

            upload_pass_form_field "^submit$|^description$";
        }

        # Pass altered request body to a backend of 8080
        location @test {
            proxy_pass   http://localhost:8080;
        }
    }
}


Tornado (rx.py)


import tornado.httpserver, tornado.web, tornado.ioloop, os.path

class requestHandler(tornado.web.RequestHandler):
print "Python receiver running"
def get (self):
print "Someone accessed through Get"

def post(self):
print self.request.arguments

#defines application of Tornado
#Pass any requests (post/get) to this handler 
application=tornado.web.Application([
    (r'/upload',requestHandler),

],)


#start the tornado, listens to 8080
if __name__=='__main__':
    application.listen(8080)
    tornado.ioloop.IOLoop.instance().start()


====================
1) Type: python rx.py
2) Goto http://localhost:8080/upload, get back to check if terminal shows "Someone accessed through Get" -->to show python working properly
3) Type: sudo nginx
4) Goto http://localhost/   --->should show an upload form
5) select file, upload ---->should show a blank page
6) go back to terminal, python interpreter should show something like this:

{u'file1_content_type': ['image/jpeg'], u'file1_md5': ['7fb958cd71e98537dd8e751d16f04ab9'], u'submit': ['Upload'], u'file1_name': ['1 copy.jpg'], u'file1_path': ['/tmp/9/0040448919'], u'file1_size': ['209926']}

1 copy.jpg is my uploaded file name. The key "file1_name" is the name of your upload form + suffix (_name) added by Nginx.

PS. Oh the file permission stuff may cause troubles. Pls change to 777 by chmod for testing purposes (remember to set to a safe value before into production!!!)

==End==

No comments:

Post a Comment