Reading Oracle DB Schema?

Apr 30, 2014 at 11:23 AM
Hi all,

Great lib!

Ive been trying to use the reader against an oracle DB, it is quite big, 5k tables, 5k views circa, and I am using the schema owner of these objects to perform the read, but the read always fails (after memory runs out).

Once I execute the read, there is a period of say 10 mins where there is data being collected from the DB (network activity on local machine), but then task manager shows no further network connectivity, yet memory usage and CPU usage continues to grow and grow until the machine runs out of memory and the app crashes?

Reader works fine with using schema owners who own less objects.

But I'd really like it to work with the main schema owner.

Anyone have any ideas, or pointers, or whether I'm doing something wrong?

Puzzled why the read takes so much memory yet stops reading from the DB quite quickly?

Thanks all!
Apr 30, 2014 at 3:50 PM
After the schema is read into memory as datatables, there's a post-read process where the model is turned into a rich object model, and then it links up all the references. For instance, bidirectional links between table and column, and between indexes and constraints.

The rich model is great and useful, but that's several times the memory of the original plain datatables, which won't have been disposed until it's all finished.

Over a few hundred tables that's still workable- but it looks like you've hit the limit. 5k tables + 5k views within a single schema is pretty big.

If you're using databaseReader.ReadAll(), it's loading stored procedures and packages (and sources) as well, which in an average Oracle database is pretty huge.
ReadAll() does several calls that are available individually- AllUsers(), AllTables(), AllViews() and AllStoredProcedures() (the latter is functions and packages too).
You can try just with .AllTables() - it also does constraints + indexes so it's still a rich model.

If you still hit problems with the smaller model, there's a lighter way of working with the API. databaseReader.TableList() which just retrieves a list of table names. Then you can hit databaseReader.Table(tableName) for specific tables. Loading the model this way means it only has specific tables, and you can't navigate across foreign keys if the other side isn't loaded. This API might be a better fit.