[ale] Indexing and searching very, very large Maildir

[Jason Fritcher]

On Jul 18, 2008, at 11:06 AM, John Wells wrote:
> We have a very large directory in Maildir format that we need to
> index, search and then view messages within. Large as in 672 Gigs and
> growing. We used to use mairix, but it started choking at around the
> 200G mark.
>
> Does anyone know of a tool, either open or commercial, that can do
> this effectively? We have a budget for it...


I don't know if you have any development time in your budget, but  
there is an excellent index/search library named Xapian that can  
handle what you want. If you are familiar with gmane, their search  
engine is built on top of Xapian. If not, check out www.gmane.org and  
see how much mail they're handling.

If you're interested in that route, I know someone who is really good  
with Xapian that is looking for some extra work.

-- 
Jason Fritcher
jkf at wolfnet.org