Tuesday, 16 November 2010

Automatic ID assignment in a distributed environment

Just released a new blog post on Zanopia regarding a distributed computing algorithm: The idea is to find a way to assign identifiers to nodes in a distributed cluster without using randomness. The problem explained here.

Friday, 1 October 2010

Accessing S3 buckets with libdroplet


Just released a new blog post on libdroplet, a high performance C library to access Amazon S3 like services:

http://blog.zanopia.org/2010/10/accessing-s3-buckets-with-libdroplet.html

Wednesday, 22 September 2010

Scality Launches Open Source Cloud Program with $100,000 Incentive Fund for Software Developers


Scality, the company I work for, announced plans to open-source the Software Development Kit (SDK) of its RING technology. As a kickoff incentive, Scality is offering contributing developers bounties from a $100,000 USD fund.

Scality is publishing an open-source library called Scality Droplet, which makes code immediately available for download. Scality Droplet will enable developers to easily build applications which interface with Scality RING, with Amazon’s S3 API and more generally with any object storage technology, thanks to Scality RING’s modular design.

Scality RING is ideal for applications handling enormous volumes of user-generated content such as email in the cloud or social applications, like Facebook, Twitter and YouTube. The Scality Droplet library implements common services such as encryption, compression and large file slicing to let application developers focus on their users’ needs.

The contest or grant opportunity is called the “SCOP Bounty Program,” and it will enable talented developers to apply for $1,000 to $10,000 bounties from a pot of $100,000. To be eligible, the developer’s software application must be built using code freely available from the Scality Droplet library. Interested developers can visit http://scop.scality.com for more information or to download an application form. The submission deadline is November 30, 2010.


Tuesday, 7 September 2010

ACID transactions on a distributed shared nothing system

New post from me on Zanopia blog:

It is possible to guarantee ACID properties on a scalable database system, with the key notion that reliability is pushed to the storage system, assuming that this storage system is reliable.

http://blog.zanopia.org/2010/09/acid-transactions-can-exist-on-a-shared-nothing-system.html

Friday, 25 June 2010

Evolving data storage models: beyond metadata + data separation

The metadata is the companion data of the data itself. Generally smaller, it contains some important additional information, e.g. (time of last access, access rights, folder which file belongs to, etc).

In a standalone computer system, data and metadata are stored together on a file system on the locally attached hard disk. The metadata is stored generally in the inode of the file.

In a clustered storage system, the current trend is to store data in a storage exclusively dedicated to storage and to store metadata in another cluster dedicated to DB and indexing.


The question is: is it possible to store both metadata and data in the same cluster ?

Tuesday, 24 April 2007

What are the needs for WebOSes ?


I was just playing with YouOS and wondering what are the needs for such an OS (if we can call this an OS, this is another debate).

1/ One first property is that it can be accessed from any web-2.0-capable device in the same manner: from the office, at home, from a mobile device.

2/ Another important thing is that people don't need complex operating systems any more for simple operations: listening to music, watching videos, browsing the web, chatting (see my previous article on linutops).

WebOSes might be the future of desktop computing. Anyway we have to mention that people won't like their datas to be stored in a remote server: passwords, bank account numbers, illicit mp3's, etc, so we have to find a solution for this.

Thursday, 19 April 2007

Analogy between The Google OS and a traditionnal OS

Feature/Concept Traditional OS Google OS



CPU
Amazon Elastic Compute Cloud
DMA
Memcached
Interrupts
Google alarms
File management File system Distributed file system (e.g. DHT), Amazon Simple Storage Service, Google (secret) distributed filesystem
File search Agent Ransack Google search engine
Network protocol TCP/IP Stack P2P streaming
Pipes Unix pipes, named pipes Yahoo pipes, Microsoft Springfield
Conversation Unix talk Google Talk
Web server Apache Amazon's Simple Queuing Service
User management Unix/Windows XP accounts Google centralized accounts
Window manager Windows, X Window Netvibes, Google homepage
Calendar Outlook Google calendar
Word processor MS Office, OpenOffice Google document
Spread sheet MS Office, OpenOffice Google spreadsheet
Presentation MS Office, OpenOffice Google presentation