Mar 21, 2008

Mirrors project public start

Hurray! We completed the PHP scripts for web proxies (anonymizers).

The story started 3 years ago when I published the code for web proxies on the The code became popular, I received hundreds of emails.

Then I approached the idea more seriously and launched a web proxy site -
It worked unexpectedly well - the popularity was going up by 10% per week without my attention until it overloaded the server (pretty weak one, need to mention).
Optimization allowed the site to process 1-2 thousands users per day with 2 GB of daily traffic. In about a year due to personal reasons I turned off the site.

Now I restore the project with much bigger expectations.
The code written will be published as open source, so everyone can download it and set up a good web proxy site.

I will launch at least 10 sites for myself which will point each other, creating a whole network of web proxies.
Later other people willing to join a network will be invited.

The current plan is to reach 100 000 users per day.

First site is launched by an old domain:

Tanks for attention, I will keep posting about the progress.

Jan 7, 2008

gCurl package - PHP classes to help using CURL

I prepared and published a package that helps me writing complex scripts with HTTP request/response.
You can get it from my site (gCurl.tar.bz2)
or from PHP Clases when it will get published.

I used it to handle HTTP requests/responses in a number of projects, like "PHP Server-side browser", "Craigslist submitter", in spiders and crawlers, and now it will be a part of my Mirrors project.
Now I updated it for PHP 5.2, commented generously and added a number of samples of usage.

Briefly, the package implements commonly used routines of preparing HTTP requests and parsing server response.
For request it provides means to send cookies, custom headers, GET parameters and POST data.
Response processing includes processing HTTP headers, including multi-line headers, cookies parsing ( representing them as an array convenient to work with ).
The most powerful important feature of the package is an interface that helps assigning handlers for response headers and body.
Assigning a handler for a response body allows processing the data on-the-fly, by chunks, as received from the server.
This way one can process HTTP response body without waiting for all data to be received and not consuming memory for the whole response body.

Most important, one can assign the body handler after processing response headers. This allows assigning a handler depending on Content-Type or cookie.

This allows me to write very flexible scripts dealing with HTTP.
Hope it will help you as well :)