August 13, 2007
IO – Distributed Processing
This page is obsolete. Current versions of AmiBroker feature built-in non-exhaustive, smart multithreaded optimizer and walk-forward engine.
For the purposes of running distributed processing with IO, an IO server need be nothing more than another Windows 2000 or above machine on the same local area network. Hereinafter the machine actually running IO will be referred to as the Client and all other machines as Servers to the Client. This is sort of bass ackwards in terms of how one normally thinks of a Client / Server relationship where one Server typically serves the needs of many potential Clients. Here we will have potentially many Servers at the beck and call of one Client i.e. the one running IO.
As can be seen from the chart of relative run times below, for a relatively fast single tradable system, optimizations of the same system on zero to nine additional machines result in huge gains in productivity by utilizing additional machines. Results will be even larger when processing Watch Lists as the amount of overhead drops relative to the amount of time required to process an optimization generation.
In general IO uses Windows sockets for all communication between the Client and Servers where a small IOServer program runs awaiting orders from the client, but will also use shared disk to move large amounts of data like symbol databases at the beginning of new runs. The setup is very simple and can be performed by anyone who knows nothing more about networking then how to connect two machines through a router or switch. Below is a block diagram of the typical setup and interaction:
IO also handles the following potential issues:
– Different Machine / CPU speeds are dealt with by a routine that will dynamically balance the load from one generation to the next between the client and servers to ensure that the most productiveity is obtained from all participating machines. This can be seen in the screen scrape below with the Servers ( Work Flow ) window open showing the load allocation by machine and relative optimization times.
– There is no need to duplicate databases from the client to the servers as IO will automatically perform this function saving data to its own database on the servers thus not interfering with whatever databases the user may already have set up.
– There is no need to manually adjust AA Settings on the servers as IO will clone the settings in play on the client to the servers as part of its own automated initial setup for client / server operation.
These are advanced features in IO.
A shareware version of IO with full documentation can be found in the AmiBroker Files Section …
http://groups.yahoo.com/group/amibroker/files/IO.zip
Filed by Fred at 5:10 am under Intelligent Optimization
Comments Off on IO – Distributed Processing