What to do with large volumes of data?

Question

Hello:

We are planning on purchasing Confluence for our IC design team as a document repository.

We also have a test group that generates large volumes of data and we need a repository for that also.

Is Confluence a reasonable solution or should we go with a dedicated database like mySQL?

Thanks,

-Dave

Jason Plumhoff · Answer

With Confluence, you could potentially build live charts and graphs using Bob Swift's SQL plugin to access data in your database.  But for the sheer volume of data that I think you'd produce (and the fact that you probably need to give your users the ability to define queries and interactive parameters), I suspect Confluence might not quite be what you're looking for.  It's a fantastic platform for sharing and collaborating on documents.  But while it CAN be used to mine databases, that isn't it's primary function.  If I were in your position, I'd probably be searching for a purpose-built package for that.  On the other hand, if you're trying to keep costs down and know SQL really well, I'd be looking more at MS SQL server with SQL Server Reporting Services.

Note: This is all just my opinion (and I'm well outside my comfort zone here), so take everything I just said with a BIG grain of salt.

Jason

David_Huffman · Answer

Hi Jason,

It's IC test data which will require "slicing and dicing" such as hystograms, fft's, plotting, statistical analysis, etc.  We want to be able to access the data in a similar way as a database.  The question is whether we should opt for a separate database package or can we pull it all together under one roof with Confluence.

Thanks,

-Dave

David_Huffman · Answer

Hi Jason,

It's IC test data which will require "slicing and dicing" such as hystograms, fft's, plotting, statistical analysis, etc.  We want to be able to access the data in a similar way as a database.  The question is whether we should opt for a separate database package or can we pull it all together under one roof with Confluence.

Thanks,

-Dave