Results 1 to 5 of 5

Thread: IMAPStore Replication/Failover IDEA - Multiple Virtual message stores

Threaded View

  1. #1
    Join Date
    Jan 2006
    Posts
    33
    Rep Power
    9

    Default IMAPStore Replication/Failover IDEA - Multiple Virtual message stores

    R A I S - F S

    IMAPStore - Redundant Array of Inexepensive Servers File System


    So I've been in search of a good failover solution for an IMAP server for some time now. Basically to have a somewhat realtime hot backup of my server that I can failover to.

    The problem is alway replicating the file system in some way while keeping the header and envolope information replicated to a remote server.

    So the idea is to create a sort of RAID-1 array across multiple servers for a message store. Sort of a simple GFS...

    I was thinking that we could pretty easily build a system where all the message "writes" to filesystem could actually be written to multiple machines. Their locations would be stored in a simple association table to the message-id. Then any request for a message would just do a quick lookup to find the message store. Obviosly this is a bit simplified.

    - store a message
    • store message file on multiple backend message stores 2 to n
    • store message physical location (messageid, server, path) in meta db
    • add message stores as storage is running out

    - get message
    • request message via messageid
    • lookup messageid in meta db to get physical location
    • grab message and deliver back into imap server exactly as it is now


    The idea come from mass image storage systems and Google File System. These work in a similiar way. Essentially any file is stored in multiple locations... its location information is stored in a database. When the file is requested it finds the file in storage and returns it. It could be on one machine or hundreds.

    If a system goes down the database with meta location information has already been replicated via db replication. When the file is requested from a server that is down it just grabs the next record for the "other" location on another server.

    Why?
    • Can add infinite number of servers to expand storage
    • Can replicate to multiple data centers pretty easily
    • Very cheap implementation, all software no SAN or expensive hardware
    • Total Failover.... can setup monitor that assures that messages are
      replicated to as many machines as you'd like
    • Tested, this is how we developed the mass-image storage at our company. This is essentially how Google File System works minus splitting files into blocks.
    • Should be very pluggable into Zimbra... meaning make a chance to the MessageStore object and be done with it... the rest of the system shouldn't have a problem.


    Anyone interested...?
    Last edited by kbaker; 07-20-2006 at 05:16 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •