Dec 26, 2007

What you don't know about USSR

I met an interesting article about the look of normal western people at our traditions and at the USSR, the country I was born in.

It tells some nice story about change in the culture and society happened after the fall of USSR.

How did the Richest Russians become Rich

Nov 19, 2007

harm of the indian code

if($pay == 3){ $pay = 3; }
else{ $pay = $pay; }

If it's "three" - let it be :)

Nov 17, 2007

vertical block in HTML and CSS

I met a task: to create a vertical image (line of repetitive images) starting under a header of a page and ending at the bottom.
If the window size of the browser changes the line should still hit the end of the page and no vertical rollers should appear.

After spending a day I found a solution. Take a look at the vertical bar to the left:



HTML:

<div id="vertical"><div></div></div>

CSS:

body{
height: 100%;
}
div#vertical{
position: absolute;
top: 0px;
height: 100%;
width: 56px;
overflow: hidden;
}
div#vertical div{
background-image: url('/images/vertical_bg.gif');
background-repeat: repeat-y;
margin-top: 212px;
height: 100%;
}


Explanation.
Here I use two DIVs, one in another.
The first has a full-page height.
Inner DIV has a background image and shifted relatively to the parent one with a "margin-top" style parameter.
"overflow: hidden" parameter of the outer DIV cuts the outstanding edges of the inner DIV and the browser does not have a vertical scrolling.
This way it always fits the page heights and does not cover the header.

The outer DIV has an "absolute" positioning.
Of course, I should remember to reserve the space to the left for all elements on the page with a "margin-left" or a "left" CSS parameter.

Nov 15, 2007

VPS - a good choice for hosting new web projects

It is convenient to run a web project on a personal server rather then on a public hosting. Reasons are:
  • Custom configuration - developer can provide a more productive and efficient solution not limited by compatibility requirements;
  • Full (root) access - convenience in management;
  • Security - in most public hostings people can access files and databases of each other, with is not good in many cases;
  • No limits - host as many web sites as you want, run your own email and DNS services, etc.
The main drawback of a dedicated server is it's cost. The fee for renting a normal dedicated server in the US is about $100 per month. For new web projects that are serving no users it means wasting money - the server capacity will not be consumed.

The solution is to use a Virtual Private Server. It feels almost like a dedicated server with a full access, but much cheaper because a real physical machine serves many VPS.

You can find many providers offering VPS from $15 to $45 per month.
Disadvantages are:
* Low memory limits (usually 64-256 Mb)
* Disk subsystem is usually overloaded and very slow
Rest is fine - processor is not loaded usually, and the bandwidth allowed is more then enough.
So you'd better use caching wherever you can and not run beyond your RAM limit cause swap is very slow. If you do that - your sites will be running as fast as they would on a dedicated server.

In the future posts I will describe how I am setting up a full-functional and productive web service on a cheap VPS with low memory consumption.

Aug 21, 2007

web master price list

I met this joke on some programmers forum:

Web master services price list
  • I do everything: $100
  • I do, you watch: $500
  • I do, you help: $1000
  • You do everything: FREE!
I don't know the author, unfortunately, but "I do, you help" is something VERY right!

Aug 10, 2007

russians hackers and the cost of security

Recently I analyzed a security of a big site and found an issue that is not easy to notice, but it lets people do a bit more then they should. I did it for fun and for a "prove of concept" that I can find it, not for destruction, but it made me think.

Security is measured with the cost one has to spend to break it. Let's take a good American programmer with $150 thousands per year salary building up the system.
Let's take a good Russian programmer who makes $1 or 2 thousands per month.
It is obvious that a Russian will be ready to spend a month to exploit the system and make $4000, while an American company spends 12 thousands for a month of work of it's employee.

Why not Indians or Chinese? Well, they too, but Russians historically have a solid mathematical, engineering and computing schools, and half of major IT companies in US have Russians among founders and chief developers.

I don't really know a solution for a short period. In the long run the income will more and more depend on qualification rather then on geographical location, as Bill Gates wrote in his book.
But for now the only way for companies is to accept the problem and earn more then lose.

Anyway, good programmers respect professional growth and creativity, not cash or destruction.

Aug 2, 2007

nGinx vs Lighttpd - web server of choice

If you run and maintain a web server you are definitely interested in the web server software.
Apache is an old good friend and we all are very happy about it.
But if your load grows, you may realize that Apache with PHP eating 20-50 Mg of RAM per child is a bit too much.
This mean that you can't run more then a couple of hundreds of children simultaneously even on a powerful server. Not a big problem, how has 100 simultaneous users - you can say.
The issue is that a plain Apache server can easily be brought down.

A 5-line script opening 200+ simultaneous connections sending incomplete HTTP requests can make the server with a plain Apache stop serving requests or serve them very slowly, when the server itself will not be loaded at all.

For the cases like in the past we were puting reverse proxies in front of Apache. Now we have a better way - another web servers. Usually we choose between Lighttpd and Nginx.

The best one at the moment is Nginx - for highest speed, lowest memory consumption, ease of configuration, scalability and completeness. The negative side people find is that original documentation is written in Russian.
Here are the links for the English documentation:
Nginx wiki and Nginx - Small, But Very Powerful and Efficient Web Server

Lighty is good as well, it became famous for being used in Youtube. Especially I like the supplementary project of Lighttpd called XCache.
XCache is the fastest PHP bytecode cache at the moment and it was the first stable one compatible with latest PHP 5.2
Also, untill recently Lighttpd had the only usable FCGI process management tool. Now we have PHP-FPM (docs are in Russian as well).

Here I accidentally met someone's blog record about Lighty/nGinx comparison and hints for setting up nGinx: Nginx, my new favorite front end for mongrel cluster


Hope this links helped.

Aug 1, 2007

new virtual server

I took a VPS (Virtual Private Server) from hostingforweb.com today for my projects.
Among the first things I did was watching the security log /var/log/secure

First lines of the logs are:
Aug 1 08:34:01 server sshd[15581]: Server listening on 0.0.0.0 port 22.
Aug 1 08:37:39 server sshd[30537]: Accepted password for root from 91.149.137.55 port 2781
means, someone accessed the server as root 3 minutes after it was started

Now imagine my shock when `whois 91.149.137.55` shown the "Belarus republic" as country.
A large US hosting provider hires administrators in the "black-listed" country?
Or Belorussian hacked the servers in 3 minutes after it's installed???
I even wrote support a question.

And the fact is that they have a "Minsk office" in Belarus.
Nice, now US is a place where Russian admins set up servers for Russian programmers.
And Bush politics does not matter ;)

And should I mention, this is probably the 1st time I see the latest version of PHP is already installed correctly with Apache 1.3, which means admins did it themselves, and there were almost all development packages I usually miss in the fresh-installed Red Hat distributives.

Now I'll check how stable their network and system are.

~ August 10, 2007
the server is stable, configuration is a really good one.
The only problem I see is the very slow disk operations.
Looks like their hard disks are overloaded and it takes significant time to do anything involving operations with files. For my tasks it is not critical.

Jun 8, 2007

Jobs and Gates joint interview

I spent several hours for it - and it worth that.

An "all things digital" talk show with Steve Jobs and Bill Gates .

May 27, 2007

a new bike



I got a bike! Bright, attractive, manoeuvrable orange-black Gary Fisher SF Advance 21.5".

Sinse I got WOW-addicted and quit I found at last a real incarnation of my virtual horse.

Do you ever have sleepless nights in summer, when it's hot and stuffy, hard thoughts and expectation of a hard day that will follow? Get on a bike and ride away from the city to the lake before the sinrise. You will start enjoying the insomnia :) Proven today.
Oh my poor haunches ...

May 23, 2007

open source esoteric

In early days of open source many people treated the idea sceptically. The reasons look obvious: firstly - who will do the serious work for free, secondly - who will do the serious business on the solutions with no-warranties or reliablility?
Now we know that open source software for web often has higher quality code, is more reliable, flexible and faster then the commercial alternatives. But why?

I don't know for sure, but I will make one strange assumption based on my own experience.
When I prepare and publish (say, give away for free) some interesting solution, I always get back more. I get emotions, feedbacks, attention, recognition.
It feels like the world gives me back what I need - or maybe I become ready and open to get it?
And I always get new interesting projects. And always want to do and publish more!

I don't know why, or how, but world does not obey the laws of the capitalism. As a professional economist (bank officer in the past), I disagree with many laws of the classical Smith's economics.
The world lives on it's own, and if you ever try to publish some good open source code - you'll never regret and only think of how to publish more.

May 21, 2007

Easy professional emails

I had a task to send some emails from the site scripts in the HTML format, while other emails should be kept in the plain text.
A big deal - what's difficult in sending emails? Nothing, actually.

I had a class in PHP that used Smarty template to fetch letter body with subject and send it either as HTML or in plain text using the 3rd party PHP library.
I rewrote the wrapper and made it usable for all kinds of tasks I have in my web practice.
This are the registration-confirmation-congradulations, invitations, monthly newsletters and notifications.

Technically, it requires sending a relatively small amount of letters from a PHP script with the content created from a template. The template has to be editable by the non-programmers
(to let a customer do it himself).
The goal of rewriting the script is to forget the endless options I had to code for sending emails. Just make it simple and do the work, but if anything beyond is required - the full power of both packages is available.

Yesterday I wrote samples of usage for my script, compiled it in a working package and published it on phpclasses.org.

References to the packeges used.
One is the "MIME E-mail message" library by Manuel Lemos. It allows sending emails from a PHP script with attachments, in plain text, HTML-formatted, with images, with non-latin characters, bulk-sending, etc ... I just can't think out what it does not allow about sending emails.
Another is the Smarty template engine I use in all sites I do. It is convenient to prepare the email body and subject with the same template engine as used for generating pages output.

May 14, 2007

Hotmail "Not found"

I think everyone who cares about IT saw an announcement about "Microsoft launching new Windows Live™ Hotmail ®"

But if you go directly to http://Hotmail.com/ (without www.) you may see "The page cannot be found" in the background behind the login window. And this lasts for weeks now.

Note - some people see the pages correctly, it's not clear why others don't.

May 10, 2007

US hunt down E-gold

E-Gold Indicted for Money Laundering

Pity, it was a convenient and fast way to get money.
Yes, it was a favorite payment system for fraudsters, casinos, and others. But it was a good way for hundreds of thousands of programmers in Europe and Asia to get payments from the western customers.
Will there be less fraud when e-gold get closed? I am sure it will not.
But US government decided to kill rats by burning the house.
Now we (web developers) got one more problem in our lifes.

May 8, 2007

Sea, sun, wine and girls

I had a vacation - 3 days by the sea!
With 2 friends I went to the south for 700 km (435 miles) by a car.
That was sun, fun, and cool.
Other day it was cool literally - rain and clouds could be depressive. Fortunately friends were alongside :).

So here are some pictures!

P.S. For those who wrote me that "girls topic undisclosed", specially adding a picture of a young accountant from the beach:

Apr 8, 2007

AJAX - beware! (of illiterate journalists)

Today I read an article in the informationweek site that describes a totally fatal security issue in Web 2.0 techniques existing in all popular frameworks.

I think, maybe I should hire some student girl able to write articles?
I will give her a set of keywords on web development subject and ask to write 20 articles proclaiming problems and close fall of google/yahoo/ms/web2.0/youtube/myspace/widgets/vista/open source
Some kind of the end of the world forecasts in IT...

The most importaint for such articles is to have killing titles, all figures should be per cents from some other data that is not provided.
E.g. "60% of the open-source widget frameworks are unstable and provide critical security issues".

Seriously, it may work ... not as a source of info of course, but if I place ADs all around the page and link articles to each other, it will be a nice form of a doorway :)

Apr 2, 2007

professional web developer characteristics

I met a wonderful article:
What separates a professional PHP web developer from a scripter?

It is probably the first case where I see a definitive list that can be used to make a formal test and make a rather correct descision based on it - whether the PHP developer is a professional or not.
I consider it much more adequate then those BrainBench tests with questions on some strange topics that a real developer will never meet in practice.
(I got some master sertificates on brainbench in web technologies, think it's a waste of time)

Mar 28, 2007

As I expected in the previous post, all the shouts of the journalists that people downloading tons of video will overload bandwidth capacities of Internet Service Providers are ungrounded.

IBM presented a new fiber optical chipset that allows to transfer 160 Gigabits (20 Gbytes) per second.

There is also a nice article from Reuters about Web 2.0 startups funding.
They writes that investments in web 2.0 startups are not high, but doubled last year: $844.4 million for 167 firms in 2006, up from 95 companies year before and just 35 in 2004.

Mar 18, 2007

new york times for lamers

A customer of mine sent me an article from New York times: "Popularity Might Not Be Enough". Along 3 screens of text they chew that life is hard, internet is not a place where money fall for doing nothing, and "for a general-interest site to generate $50 million" per month the site has to have "billions page views a month". Did I miss something or they are just comedians?

Especially I laughed over "new technologies that could allow I.S.P.’s to identify the biggest bandwidth users". Technologies to track statistics of traffic consumed per user is something new?
If ISP sells "unlimited traffic" package - it is not bound to bandiwidth. If it's "to identify the biggest bandwidth" - it is not "unlimited". In the worst case unlimited packages will become more expensive and ISPs will offer the "per-GB" packages again.

Why do people waste so much time for freebie hunting and fake fears ... infantility?

Mar 11, 2007

PHP security settings

Sometimes there is a need to set up a 3rd party script, like a forum, on the dedicated server I am responsible for.

Maybe you remember, a couple years ago a serious security issue was discovered in PHPBB, a very popular forum software at that time, and hundreds of thousands of servers all over the world got infected by a worm. I do remember that case.

So there is a question - how to use a 3rd party script that you don't trust in full?
There are several options I recommend to use upon compilation of PHP, in php.ini and in the VirtualHost section (for some sites).

Here are some settings I personally use in VirtualHost (configuration of Apache web server) to run the potentially unsecure application:



<VirtualHost *>
...
php_admin_value upload_tmp_dir "/path/to/upload_tmp"
php_admin_value open_basedir "/
path/to/forum:/usr/local/lib/php"
php_admin_value disable_functions "shell_exec,exec,system,
passthru,proc_open,popen,curl_exec,pcntl_exec, socket_create,socket_create_listen"


</VirtualHost>

(Note: I have the disable_functions value written in one line without spaces)

There are some security-related settings I have in php.ini on the production servers (PHP 5.2).

;turn on for the sites I need in the per-host config
register_globals = Off

;does not really safe, but too restrictive IMHO
safe_mode = Off

;notices almost always tell about more serious problems
error_reporting = E_ALL

;it doesn't save from the SQL injections anyway
magic_quotes_gpc = Off

;don't allow to execute arbitrary code as a loaded module
enable_dl = Off

;Anti-DOS settings
max_execution_time = 30
memory_limit = 16M
post_max_size = 8M
upload_max_filesize = 6M

;before PHP 5.2 (when allow_url_include was not available) I had it "off"
allow_url_fopen = On
allow_url_include=Off

;Never use /tmp, critical projects may need a completely separate storage
session.save_path = "/home/www/sessions"

;I don't allow session id in URLs
session.use_only_cookies = 1


When I set up (compile) PHP on the server, I compile the web server module and the CLI module separately with the different options of the ./configure command.
For the web server module I add "--disable-posix --disable-sockets --disable-ftp --disable-sysvsem --disable-sysvshm --disable-shmop --disable-pcntl"
For the CLI module I have these options "--enable"d.

These settings provide me with almost unlimited flexibility of the the dedicated server environment and good security while running my code (not recommended for the public hosting).
The untrusted code I run as the separate sites, adding the "disable_functions" setting you can see above in the VirtualHost section.

Mar 5, 2007

spring flowers

Today I bought flowers! It is unusual - yet very exciting :)
Here they are - images of a web cam quality, but I enjoy them!



Feb 21, 2007

sysem design error

About a year ago my team has done a site MyActivityMatch.
One can use it to find who to play sports with.
The system has a rather complex search algorythm - it takes into account geographic location, distance, date, time (hours or a day part like "morning"), time zone, sport type, skill.

I underestimated the complexity, defined the lower price then it deserves, and entrusted the developement to an employee of mine, a good person and a good programmer, but not too experienced with the complex solutions at that time.

After a while it came out that the system works slowlier then expected.
Some bottlenecks were fixed close after launch and the site continued to work.

Some time ago the customer ordered the modification of the system.
Being knowledgeable about the complexity I took the work myself and started to solve each of the problems I saw, one after another.

Couple of days ago, after several months of fighting with problems, I faced an unexpected issue - the database contains many duplicating records. ID of the records are different, but the data is the same.
It's not a bug, it does not break anything. It just makes the system slowlier.

I call it a system design error. The developer did not foresee the interactivity of elements and modules. He coded the solution per specification - and did it good!

Whose fault is it, who is responsible? Me and the customer are.
Me - because I incorrectly positioned the project and provided the solution with a logical issue. Now I have to waste time to rewrite parts of it from sketch.
Yes, I kept the budget and fulfilled the contract per specification.
But is it what the customer wants?

Customer - cause actually everything in the project is his responsibility.
He did not provide any requirements for the performance and inner structure, never discussed anything, ignored any offers and was just holding the idea to get the cheap solution of the complex project.

A consequence I made for myself: Do not avoid system design, as you don't avoid the layout design, accept it's value and cost.
There are thousands of people ready to help a customer spend his money, but I want to do something valuable besides cash.

I hope I won't get into such pitfalls in the future.
At least during last year I havn't - I set the adequate price for my work and provide the really good solutions.

One can't make a million dollars by paying a couple of thousands, doing nothing and expecting a cool programmer will code something great. I don't even understand why so many people expect this :)

Feb 20, 2007

web 2.0 radio - plays what you want to listen

At last I see a streaming solution I really like.
Pandora plays the music I choosed to listen. I am not dependent on the DJ mood, don't have to switch channels when hear the commercial and may skip the track I don't like.
I gave a name of a band I wanted to listen and it made a playlist of the similar music - so I don't choose each song, and still listen what I want.

If you got a broadband connection - try it. Good implementation of Web 2.0

Feb 15, 2007

IT management - what programmers want

Recently I had a project to build a restaurants reviews site.

We successfully did 25% of the work during a week. Suddenly a customers said to stop, throw out the code, take a ready-made constructor PHPFox and go on customizing it.

His idea was that to buy a ready-made code customize it will be faster and cheaper then writing the custom system.
Despite I said it will be much faster to complete the system, he just wanted to get the existing additional functionality of PHPFox instead of developing (and paying for it).

I choosed an option to discontinue the project, keeping the small advance that did not cover the amount of work done.
The most importaint reason was that I don't want to work with the old platform.
PHPFox is an out-of-the-box solution, intended to be sold to as many people as possible. It has to be compatible with older releases and old platform - so it keep an old ideology.
That's not what I like to work with! I like to do something new :)

Today I saw a great article Nine Things Developers Want More Than Money

It is great. If you plan to do a successfull IT project, you will depend on your developers, and you will need good reationships with them. Please read the article, cause it's very true.

Feb 8, 2007

Today I saw the description of an IT project starting with the words:
The requirments are still in the works. I am looking for someone to take this project from the craddle to the grave.
Oh yes, there are lots of people ready to bury the project! :)

Feb 3, 2007

video conferencing

Half of the projects I am discussing last month is connected with online video streaming.
Yes, YouTube has changed a direction of thoughs of many people with it's price :)
Pity, but most of people who want to do something like that just emphatically keep wanting a freebie - to make a million paying a thousand or two.

What I recommend to look at is a site Stickam.com. I enjoy their video chat feature.
If you have a web cam and a good internet connection - you are ready for web conferences.
Try - it is great.
Now I can discuss projects using a video conference with my customers.

web, control over the business and open-source

A customer sent me a link to an interesting article:
Zend seeks a sustainable open-source model

I think it has a couple of interesting ideas.

1. Many businesses based on the open-source double the revenue each year.
That's a good and stable trend, and I saw better results for the correctly managed web projects.

2. An old generation of IT businessmen still can't realize what to do with the open source.
I mean the phrase
the company does not have "complete control" over the project
Control, control, control ... world is divided by the spheres of influence between USA and USSR ... companies own their means of production ...

Time to forget that.
In IT it's all a matter of user satisfaction, flexibility and speed.
The essence of a "byte" is being transferred, it does not exist in a static form, same as light.

I don't even know how to approach the flexibility of the open-source from the "control" concept.

Jan 24, 2007

AJAX, javascript libraries and ideas

Last months I use JavaScript very intensively and found a very convenient and efficient approache that I did not read about.

You can see many Javascript libraries around, at least 3 of them are well-established and popular:
Prototype (with script.aculo.us), Yahoo! User Interface Library and a Google Web Toolkit.

I personally don't like to watch a progress bar while those bells&whistles of a web 2.0 site are loading. The only way to save visitors of your site from a tedious waiting is to count the size of your web applications and place efforts to minimize it.

Here are my ideas on how to achieve great effects on the site, save bandwidth and development time.

* One idea I found to be very efficient is that one library is not enough.
Good results can be achieved when using Prototype as a base helper library and the DOM Tooltip to draw menus.

Developer can combine best parts of different libraries to achieve better resulsts.
For example, I like YahooUI for it's great ready-made animation effects.
But for AJAX, DOM, Events and general DHTML development I prefer Prototype 1.4
Yes, Yahoo library has means to process AJAX and events, and script.aculo.us has many great features, but I like them less them.

* Second idea - JavaScript can be optimized before publishing.
Consider processing your .js files with a tool like JSMIN before uploading to the production server.

The first advantage - file size is reduced by 15-30%, and prototype 1.4 becomes 35 Kb instead of initial 47 Kb.
Second advantage - you can format your code, write full detailed comments, and not suffer from filesize penalty.

Unfortunately, not all javascript code keeps working after being optimized by JSMIN, so check first.

* Third idea is to take HTTP into account.
Developer can foresee and utilize caching abilities of the browsers.
One can also try to foresee the utilization of the keep-alive connections.

If you place a relative URL to the library - there is a chance it will be loaded in the same keep-alive connection with another HTTP request.
If you have subdomnains, you can use an absolute URL and the cached pre-loaded library will be immediately available on the pages of the subdomains.

To be continued! There is so much to write about AJAX :)

Jan 19, 2007

Security

Once I had a conversation with a customer about security.
He sent me a link to some article, but I decided not to read it.
Talking about it I understood that I need to explain - what the security is when it comes to the web development and AJAX in particular.

I asked for the permission to publish the chat, and here it is:


...
Gri (12:11 AM)
AJAX is the client-side technology.
The security of the web system does not rely on the data received from the client (browser), if developer is assuming that the data as dangerous always.

customer (12:16 AM) :

yes. although it is not our first email about it.... we don't doubt you.

Gri (12:16 AM) :
There are several major security problems in web applications.
This is not written in articles for some reason (maybe I should write one?)
- server security (OS, firewall, non-web-related issues)
- web application security (scripts issues)
- cross-server scripting vulnerabilities
- DDOS

Gri (12:19 AM) :
The first is a server/cluster configuration, it will not be a problem.

Second also separates by different categories, from application design to sql injections and unknown platform bugs.
This is what I don't worry about also.

3rd is a "hard-to-forsee problems" category.
How and in what context will some people want to exploit somebody's security - I don't know.

Gri (12:21 AM) :
I can tell a good story, how people were stealing ICQ numbers

customer (12:22 AM) :
I'm sure that you've seen a lot from behind the scenes
we expect a lot of incoming troublemakers.

Gri (12:22 AM) :
Sometime ago hotmail made accounts expire in 3 months;
Lots of people regietered ICQ with hotmail addresses, hackers found this expired accounts, registered new ones in place of expired with the same hotmail addresses - and got the passwords...

Neither hotmail nor ICQ is directly guilty, as you can see.
Just many months later hotmail fixed a problem - the web mail systems don't allow to re-register an expired accounts anymore.

Gri (12:25 AM) :
The last issue is DDOS. This is a problem in general, and noone has a uniform solution.
Last month e-gold (a big payment processor) was periodically offline for several days day due to DDOS attacks.

customer (12:25 AM) :
but, everyone is aware of it and working on it, correct?

Gri (12:25 AM) :
I never practically experienced a really hard DDOS attack.
Though, I keep it in mind and don't leave the obvious performance bottlenecks in the system.

The only solution is monitoring, detecting attacks and fighting against when it happends.

customer (12:27 AM) :
when more and more applications move to database and become popular... more and more people will try to attack things.

Gri (12:27 AM) :
People don't attak themselves. The hackers attack through controlled computers.
Most serious attacks are from infected computers.
But they are rare, as serious epidemies are rare and hard to organize.

new blog

Hi all,
here I am starting my new blog on web development.
There will be notices, short articles and other staff I want to get published, but don't want to bother placing it on my main site :)