Saturday, December 22, 2012

Communication between Nginx upload module and Tornado, Python

Basic communication flow:

User (Internet, Port 80)===upload===>Nginx (listens 80) saves files===redirect modified POST headers via port 8080===> Python Tornado  (listens to 8080)====> return users information

1. Setting up Nginx

- I put the upload form (index.html) on /var/www/html/php, so I set this as the working directory (bad naming, but nevermind!)

==========
Index.html (from the Official example)


<html>
<head>
<title>Test upload</title>
</head>
<body>
<h2>Select files to upload</h2>
<form name="upload" method="POST" enctype="multipart/form-data" action="/upload">
<input type="file" name="file1"><br>
<input type="file" name="file2"><br>
<input type="file" name="file3"><br>
<input type="file" name="file4"><br>
<input type="file" name="file5"><br>
<input type="file" name="file6"><br>
<input type="submit" name="submit" value="Upload">
<input type="hidden" name="test" value="value">
</form>
</body>
</html>



Nginx.conf:



worker_processes  20;

error_log  /var/log/nginx/error.log debug;

working_directory /var/www/html/php;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    server {
        listen       80;
        client_max_body_size 100m;

        # Upload form should be submitted to this location
        location / {
        root /var/www/html/php;
        index index.html index.htm;
        }
        location /upload {
            # Pass altered request body to this location
            upload_pass   @test;

            # Store files to this directory
            # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist << You must manually create these subdir before you upload, because Nginx upload does not support creating new directories

            upload_store /tmp 1;
         
            # Allow uploaded files to be read only by user
            upload_store_access user:r;

            # Set specified fields in request body
            upload_set_form_field "${upload_field_name}_name" $upload_file_name;
            upload_set_form_field "${upload_field_name}_content_type" $upload_content_type;
            upload_set_form_field "${upload_field_name}_path" $upload_tmp_path;

            # Inform backend about hash and size of a file
            upload_aggregate_form_field "${upload_field_name}_md5" $upload_file_md5;
            upload_aggregate_form_field "${upload_field_name}_size" $upload_file_size;

            upload_pass_form_field "^submit$|^description$";
        }

        # Pass altered request body to a backend of 8080
        location @test {
            proxy_pass   http://localhost:8080;
        }
    }
}


Tornado (rx.py)


import tornado.httpserver, tornado.web, tornado.ioloop, os.path

class requestHandler(tornado.web.RequestHandler):
print "Python receiver running"
def get (self):
print "Someone accessed through Get"

def post(self):
print self.request.arguments

#defines application of Tornado
#Pass any requests (post/get) to this handler 
application=tornado.web.Application([
    (r'/upload',requestHandler),

],)


#start the tornado, listens to 8080
if __name__=='__main__':
    application.listen(8080)
    tornado.ioloop.IOLoop.instance().start()


====================
1) Type: python rx.py
2) Goto http://localhost:8080/upload, get back to check if terminal shows "Someone accessed through Get" -->to show python working properly
3) Type: sudo nginx
4) Goto http://localhost/   --->should show an upload form
5) select file, upload ---->should show a blank page
6) go back to terminal, python interpreter should show something like this:

{u'file1_content_type': ['image/jpeg'], u'file1_md5': ['7fb958cd71e98537dd8e751d16f04ab9'], u'submit': ['Upload'], u'file1_name': ['1 copy.jpg'], u'file1_path': ['/tmp/9/0040448919'], u'file1_size': ['209926']}

1 copy.jpg is my uploaded file name. The key "file1_name" is the name of your upload form + suffix (_name) added by Nginx.

PS. Oh the file permission stuff may cause troubles. Pls change to 777 by chmod for testing purposes (remember to set to a safe value before into production!!!)

==End==

Friday, December 21, 2012

Installing NginX on Linux through yum

The OS I am using: CentOS 6

$ sudo yum install nginx

After that, nginx will be installed at this path:

/etc/nginx

To start, simply type: 

$ sudo nginx

If errors: bind to 0.0.0.0:80 failed (98: Address already in use), most likely due to the occupation of socket by apache. Stop apache by:

$ sudo service httpd stop

and retry.


Saturday, December 15, 2012

Hosting Ragnorok Online Private Server using rAthena

OS: Linux (CentOS 6)
Installation date: 14/12/2012
SVN: 17020 ==>This SVN uses 2012-04-10aRagexeRE.exe (we will talk about it later)
Version of RO episode: 14.1

Server Side:

Please follow the tutorial here:
*Please note that the last line of importing sql

mysql -u root -p rathena_log < logs.sql

Should be 

mysql --user=root -p rathena_log < logs.sql

Then everything should go fine.
Type 

cd ~/trunk
./athena-start start

to start your server.

=============
Client Side:

It took me a while to find the original RO game programme.
You can download RO Ep 14.1 here:
or 

Install it.

On your testing client machine (Windows), install SVN by going
download and install.

To download a patcher for generating the main RO exe file, use this ShinsDiffPatcher
open your command line in Windows, type

svn checkout https://subversion.assembla.com/svn/weetools/trunk/ShinsDiffPatcher/ <Replace here your path to download this>

You have to download a Renewal file, and patch it by ShinsDiffPatcher to generate main RO exe.

I tried this

But later I encountered errors using this. There are many people online saying having errors on server as below, and cannot login:

clif_parse: Disconnecting session #7 with unknown packet version (p:0x464d,l:19).

So, I tried other versions. This works. I suggest from the beginning we use this.

Open ShinsDiffPatcher, Choose WeeDiffGenerator from bottom menu, 
"Source executable": "2012-04-10aRagexeRE_J.exe
Choose automatically select. Untick "Read data folder first"
Then "Patch it"-->generated a exe file.

Next we are gonna generate a data file for our "in-house made exe" to use. 
Use GRF builder

Download 
(Version 3.0)

Unzip, 
Edit data/clientinfo.xml, modify the address to your IP

<address>123.123.123.123</address>

Then follow the tutorial from this video on using grf builder (use "data" folder to generate, not including folders like "AI" etc)
This generates a yourfilename.grf

In your official RO installation folder, find data.ini, add 0=yourfilename.grf so that it looks

[Data]
0=yourfilename.grf
2=data.grf

save.

Put the patched EXE and GRF file together to your RO folder. 
Start your server.
Start your EXE file. Bingo.


===========Materials with unknown use============
http://svn6.assembla.com/svn/ClientSide/Packets/Packet_db/

=====Debug======

Q1 # Can play in the LAN, not through the Internet
Reason 1) Did not forward the port if your server is inside the router
---> Forward the 3 ports 6121, 5121, 6900 to your server computer

Reason 2) Edit the Conf files
---> char_athena.conf


// Character Server IP
// The IP address which clients will use to connect.
// Set this to what your server's public IP address is.
char_ip: xx.xx.xx.xx <<your WAN IP

---> map_athena.conf


// Map Server IP
// The IP address which clients will use to connect.
// Set this to what your server's public IP address is.
map_ip: xx.xx.xx.xx<<your WAN IP

Reference to
http://rathena.org/board/topic/58572-probem-on-connection-to-server-with-wan-ip/




Tuesday, October 30, 2012

Installing OpenCV on MacOS X via MacPorts

Installing OpenCV on MacOS X is simple.

1) Installing MacPorts
i) via pkg

For me, I tried downloading the MacPorts pkg files (the claimed easiest way) from here:
http://www.macports.org/install.php

After installation you should be able to use:

sudo port -v selfupdate

to update the MacPorts. However for unknown reasons it didnt work.

ii) via source

Then I tried downloading the source, i.e. MacPorts-2.1.2.tar.bz2, extract it.
then cd to MacPorts-2.1.2 and type:

./configure && make && sudo make install

After all these things you have to edit the PATH variable to allow the terminal access the command "port". Go to /etc/, copy the "paths" file out, append the following to the end of file:

/opt/local/bin

and put it back. (This saves you from using the troublesome vim editor)
thats where the MacPorts installed. Then you should be able to access the following update command:

sudo port -v selfupdate

2) Installing OpenCV

- Problems: After setting up the MacPorts, I tried installing openCV by this:

sudo port install opencv +python27

But after installation I find

import cv

in python did not work.

I figured out that it is the problem of having many different versions of python install on Mac (Both my installation and MacOS original version)
The easiest way to fix this is to remove all the versions of Python, and install MacPorts-version of Python. The OpenCV from MacPorts only works for Python from MacPorts too.
if you have an user-installed version of python, remove it by:


rm -rf /Library/Frameworks/Python.Framework


because the OpenCV will not be configured to your original version of python nor the come-with-the-mac version of python.
Then try to type:

python

If you are still able to open python, it is possibly due to the old version that comes with the macOS. Go to /usr/bin/ , rename "python" to other names
Now you shouldnt be able to call python in terminal.
Then install python27 on MacPorts, type:

sudo port install python27

After that, type:

sudo port install opencv +python27

This will install the python bindings of openCV

Then it automatically calculates and install opencv for you. It takes about 2 hours to compile and install all necessary dependencies.

When it is done, try enter python in terminal, and type:

import cv

If everything goes right, it will do nothing.

You can now go to this github to get hello world applications.

https://github.com/jessicaaustin/robotics-projects/tree/master/opencv-tutorial

Friday, August 24, 2012

No GCC in Mac OS X Python

GCC - No such file or directory on Mac OS X in Python 2.7

Python 2.7 default requires a GCC 4.2 compiler. However the Mac OS X does not have a built-in GCC. To install GCC, we can download XCode (if you do not want such a huge thing to be installed, choose to install only the command line tool from Apple. Refer to this link.)

After that, you can go to terminal and type:

gcc -v

You should be able to see the info of GCC.
However, Python specifies GCC-4.2 to be called in the interpreter. This can be solved by changing the name of "gcc" alias in /usr/bin/ to "gcc-4.2". This allows gcc-4.2 to be called in Python. But in the terminal you should now type:

gcc-4.2 -v

to get the gcc version info.