Platform (formerly OS) Discussion

Automatic ID assignment in a distributed environment

2010-11-16T10:19:00.002+01:00

Just released a new blog post on Zanopia regarding a distributed computing algorithm: The idea is to find a way to assign identifiers to nodes in a distributed cluster without using randomness. The problem explained here.

Accessing S3 buckets with libdroplet

2010-10-01T15:45:00.006+01:00

Just released a new blog post on libdroplet, a high performance C library to access Amazon S3 like services:

http://blog.zanopia.org/2010/10/accessing-s3-buckets-with-libdroplet.html

Scality Launches Open Source Cloud Program with $100,000 Incentive Fund for Software Developers

2010-09-22T10:41:00.005+01:00

Scality, the company I work for, announced plans to open-source the Software Development Kit (SDK) of its RING technology. As a kickoff incentive, Scality is offering contributing developers bounties from a $100,000 USD fund.

Scality is publishing an open-source library called Scality Droplet, which makes code immediately available for download. Scality Droplet will enable developers to easily build applications which interface with Scality RING, with Amazon’s S3 API and more generally with any object storage technology, thanks to Scality RING’s modular design.

Scality RING is ideal for applications handling enormous volumes of user-generated content such as email in the cloud or social applications, like Facebook, Twitter and YouTube. The Scality Droplet library implements common services such as encryption, compression and large file slicing to let application developers focus on their users’ needs.

The contest or grant opportunity is called the “SCOP Bounty Program,” and it will enable talented developers to apply for $1,000 to $10,000 bounties from a pot of $100,000. To be eligible, the developer’s software application must be built using code freely available from the Scality Droplet library. Interested developers can visit http://scop.scality.com for more information or to download an application form. The submission deadline is November 30, 2010.

ACID transactions on a distributed shared nothing system

2010-09-07T14:55:00.006+01:00

New post from me on Zanopia blog:

It is possible to guarantee ACID properties on a scalable database system, with the key notion that reliability is pushed to the storage system, assuming that this storage system is reliable.

http://blog.zanopia.org/2010/09/acid-transactions-can-exist-on-a-shared-nothing-system.html

Evolving data storage models: beyond metadata + data separation

2010-06-25T17:34:00.008+01:00

The metadata is the companion data of the data itself. Generally smaller, it contains some important additional information, e.g. (time of last access, access rights, folder which file belongs to, etc).

In a standalone computer system, data and metadata are stored together on a file system on the locally attached hard disk. The metadata is stored generally in the inode of the file.

In a clustered storage system, the current trend is to store data in a storage exclusively dedicated to storage and to store metadata in another cluster dedicated to DB and indexing.

The question is: is it possible to store both metadata and data in the same cluster ?

What are the needs for WebOSes ?

2007-04-24T11:10:00.000+01:00

I was just playing with YouOS and wondering what are the needs for such an OS (if we can call this an OS, this is another debate).

1/ One first property is that it can be accessed from any web-2.0-capable device in the same manner: from the office, at home, from a mobile device.

2/ Another important thing is that people don't need complex operating systems any more for simple operations: listening to music, watching videos, browsing the web, chatting (see my previous article on linutops).

WebOSes might be the future of desktop computing. Anyway we have to mention that people won't like their datas to be stored in a remote server: passwords, bank account numbers, illicit mp3's, etc, so we have to find a solution for this.

Analogy between The Google OS and a traditionnal OS

2007-04-19T10:41:00.003+01:00

Feature/Concept	Traditional OS	Google OS

CPU		Amazon Elastic Compute Cloud
DMA		Memcached
Interrupts		Google alarms
File management	File system	Distributed file system (e.g. DHT), Amazon Simple Storage Service, Google (secret) distributed filesystem
File search	Agent Ransack	Google search engine
Network protocol	TCP/IP Stack	P2P streaming
Pipes	Unix pipes, named pipes	Yahoo pipes, Microsoft Springfield
Conversation	Unix talk	Google Talk
Web server	Apache	Amazon's Simple Queuing Service
User management	Unix/Windows XP accounts	Google centralized accounts
Window manager	Windows, X Window	Netvibes, Google homepage
Calendar	Outlook	Google calendar
Word processor	MS Office, OpenOffice	Google document
Spread sheet	MS Office, OpenOffice	Google spreadsheet
Presentation	MS Office, OpenOffice	Google presentation

An operating system in your browser

2007-04-15T17:59:00.000+01:00

Yet another hot project: an operating system in a web browser.

What is the need for it? Small businesses can now replace their expensive desktops by "linutops", minimalistic linuxes on USB keys with Firefox . So they only pay for software needed for accessing web services: e.g. Java portlets (business applications like SAP widgets, etc) running on the Web OS.

Software is Open Source. The BM is based upon packaging, support, warranty.

A video explanation here (in french sorry).

Th website here.

Seen in the video: According to the famous Gartner institute, in 2009, employees will use their own computers at work. Companies will pay for the virtualization software and compensate the employee for the hardware, a little bit like they do for the car expenses.

Haiku, yet another operating system...

2007-04-15T17:51:00.001+01:00

Haiku, as ZETA does, claims to be one of the successor of BeOS.

You can find a video presentation of Haiku at Google Tech (on 02/13/07) here

At the beginning of the presentation there is an interview of Jean-Louis Gassée, the former CEO of (now dead) BeOS. The latter says that rewriting huge software like an operating system is good because we must push technology into all sorts of directions. He also says that creating an ecosystem with combinations of alive components can lead to "unexpected results". He then compares it with the story of Google that had to convince people for the need of new kind of search engines, and then finally suceeded in.

Mickael Phipps, the leader of the Haiku Project then presents the feature of the OS:

- Unlike Linux that is generalistic (runs on desktops, appliances, embedded systems), Haiku is dedicated for desktops.

Haiku main focuses have been Responsiveness + Simplicity

Here are some various intersting features of the OS (The good):

Comes with a built-in rich API, no need to add external libraries for running cool apps).

Haiku handles multi-core and is natively multi-threaded (as they say multi-core is the future of computing).

There is C++ in the kernel : altough such practices would be traditionnaly "forbidden". Especially, such design falicitated usage of a poweful Cplusplus B+Tree implementation in the filesystem. (For LSE/OS we used BSD style macros for implementing B+Tree in C language).

The bad:

OS is 10 times slower than Linux

TCP doesn't work very well

Firefox crashed during presentation (Murphy's Law).

Gcc is not well supported.

There is not so much relationship to ZETA (altough the latter promised some drivers)

Official website of Haiku OS

Why kernel virtualization matters (a new paradigm?)

2007-04-15T17:50:00.000+01:00

Kernel virtualization is not only a fun gadget that allows to run multiple operating systems simulatenously. Today, kernel virtualization is useful for

Real time issues, by running real time code in an underlying RTOS below the general operating system.
Intellectual Property issues: moving out proprietary code from the Open Source operating system (e.g. Linux that constraints kernel code to be GPL)
Running one operating system on multiple cores or hosts.

But virtualization may be even more important in the next decade. The new paradigm is that hardware is not a constraint any more, e.g. it is possible to design some kernel with hardware constraints if emulating them on another processor doesn't add significant overhead.

There will be a time where hardware will only be the optimization of some virtualization use cases.

Kernel Designs Explained

2007-04-15T17:46:00.007+01:00

An article that explains the difference between a monotlithic kernel and a micro-kernel here. It concludes that the hybrid model is the best for today (a mix between both models). According to me the nano-kernel model (an extremist vision of the micro-kernel): "everything is out from the kernel" excepted process management and memory will appear in the next 10 years mainly for two reasons:

1/ Processing time won't be an issue any more.

2/ People won't accept that computers crash any more: Bug free or formally prooved code will be necessary. So this requires to have very small pieces of code checked independently.

Roll on your Own: