node.js - how best to 'tail -f' a large collection in mongo through meteor? -

i have collection in mongo database append logging-type of information. i'm trying figure out efficient/simplest method "tail -f" in meteor app - new document added collection, should sent client, should append end of current set of documents in collection.

the client isn't going sent nor keep of documents in collection, last ~100 or so.

now, mongo perspective, don't see way of saying "the last n documents in collection" such wouldn't need apply sort @ all. seems best option available doing natural sort descending, limit call, what's listed in the mongo doc on $natural

db.collection.find().sort( { $natural: -1 } ) 

so, on server side afaict way of publishing 'last 100 documents' meteor collection like:

meteor.publish('logmessages', function () {   return logmessages.find({}, { sort: { $natural: -1 }, limit: 100 }); }); 

now, 'tail -f' perspective, seems have right effect of sending 'last 100 documents' server, in wrong order (the newest document @ start of meteor collection instead of @ end).

on client side, seems mean needing (unfortunately) reverse collection. now, don't see reverse() in the meteor collection docs , sorting $natural: 1 doesn't work on client (which seems reasonable, since there's no real mongo context). in cases, messages have timestamps within documents , client sort 'natural order' back, seems kind of hacky.

in case, feels i'm missing simpler way have live 'last 100 documents inserted collection' collection published mongo through meteor. :)

thanks!

edit - looks if change collection in mongo capped collection, server create tailable cursor efficiently (and quickly) notified of new documents added collection. however, it's not clear me if/how server through meteor collection.

an alternative seems little less efficient doesn't require switching capped collection (afaict) using smart collections tailing of oplog @ least it's event-driven instead of polling, , since operations in source collection inserts, seems it'd still pretty efficient. unfortunately, afaict i'm still left sorting issues since don't see how define server side collection 'last 100 documents inserted'. :(

if there way of creating collection in mongo query of ("materialized view" of sorts), maybe create log-last-100 "collection view" in mongo, , meteor able publish/subscribe entire pseudo-collection?

for insert-only data, $natural should same results indexing on timestamp , sorting that's idea. reverse thing unfortunate; think have couple choices:

1. use $natural , reverse yourself
2. add timestamp, still use $natural
3. add timestamp, index time, sort

'#1' - 100 items, doing reverse client-side should no problem mobile devices , off-load server. can use .fetch() convert array , reverse maintain order without needing use timestamps. you'll playing in normal array-land though; no more nice mini-mongo features filtering first before reversing.

'#2' - 1 interesting because don't have use index can still use timestamp on client sort records. gives benefit of staying in mini-mongo-land.

'#3' - costs space on db straight-forward

if don't need capabilities of mini-mongo (or comfortable doing array filtering yourself) #1 best.

unfortunately mongodb doesn't have views can't log-last-100 view idea (although nice feature).

beyond above, keep eye on subscription life-cycle users don't continually pull down log updates in background when not viewing log. see becoming performance killer.