Under Transcode Configuration options, you can choose between 2-8 threads, for both Local and Remote.  How does MF know if/where the remote processor is?  And, can it access multiple remote processors?  We'd kinda like to get all our DS RP's involved ...

It is sort of described in the documentation. We are working on the whitepaper to describe how to set this up.

And congrats on being the first to post here. I bet you would be!  ;)

Michael

Thanks.  Unless I missed it, the docs just say to set the number of threads you want for Remote Processing, but not how to point MetaFuze to any specific system or IP address.  What's the ETA on the White Paper? :)

I know it's not me! Somethng to do with line commands and the XML stuff. I'll check with engineering to see when that whitepaper is planned.

Michael

MichaelP:

And congrats on being the first to post here. I bet you would be!  ;)

Man I was going to be me but I have been looking after the kids.

We've got a handful of XW8200s just waiting to do some remote processing.

Randall L Rike:

Under Transcode Configuration options, you can choose between 2-8 threads, for both Local and Remote. 

Also, when you are transcoding AVI, MOV, R3D, the number of local threads is automatically set to maximum although this menu will not reflect that.

OK here goes... I hope this helps

Those settings named Number of threads are actually not for remote processing at all...

Basically these settings are used to specify the number of reader threads used to read information via the parser from disk... the remote setting being for disks not phsically in the transcoding system and local for well drives that are local to the system...

So why bother I hear you say... well a number of remote shared storage solutions support multiple readers without saturating the disk subsystem. Eg. on Avid shared storage Isis or Unity we can launch a number of reader threads the gather up the data quicker. We decided to expose this to users so they can tune their systems to get optimal number of reads without "hurting" or impeding other clients which might be more critical (think RT playback or capture to tape/air). For local systems we suggest using 2 threads as this is the optimal number of reader threads we found as part of our testing.. any more and things get slower as the disk is thrashed... spending more time seeking than reading.

The second piece of information is that depending upon the file type the behavior will be diferent... for a sequence of still files think dpx, tiff, tga etc increasing the number of read threads might well help provided you do not saturate your storage with multiple read requests however for stream based parsers (avi & mov) this will not help you as they are only opened by a single thread despite what settings you might have.

For r3d files which are also stream based we ignore the number of reader threads in the UI however there is an optimization at the parser level to read multiple frames from the r3d file. The details I don't really want to get into here...

In future MF versions we are looking at making a more automagic way to identify storages, set the "optimal" number of threads without user intervention...

For setting up multiple remote render clients the idea is to write a white guidance paper however there are so many criteria which make this a hard thing to do...

Should anyone wish more details in the short term feel free to contact me directly steve.mcneill@avid.com

steve

That helps.  Is MF using CPU's or GPU's, or both?  Do faster ones help noticeably?  Does MF use all the CPU procs that a system has?  Is there an ETA on utilizing multiple computers for additionmal procs?  Our goal is to exceed Real Time prcessing if possible.

Hi Randall,

For the current processing MF is using only CPU. Over time this will change as both GPUs and other processing "domains" become available for decoding, debayering and color processing...

As for the speed of the CPU... faster is generally better however given price/ performance and other options of where to spend the same $$$ for the best system For cpu I would go for a couple of steps down from the top end processor and invest in more memory (remember that 32 bit is limited to 4Gb) and make sure that memory is properly "spread out" over the system to ensure that the cpus are "properly balanced". Also a good disk infrastructure will serve you well as this tends to be the bottleneck right now... a stripped SAS or SATA array works best and always try to store your resulting MXF file on a separate storage to the sources regardless of what disk subsystem you have as there is a cost to interleaving reads and writes... lastly if your media is remote then go for the fastest network you can afford and make it private so that you are not competing with other file-servers, data-transfers or archive systems on the "same metwork"

You can integrate MF as part of a render farm today using the xml and console version of MF as the renderer... if you have someone who is familiar with scripts it ain't rocket science.  That said there is quite a lot to getting the best performance etc in terms of network topology, storage, load balancing and integrating with other systems and any current queue manager...I will be happy to discuss this further either via this forum on 1:1

Also I am looking at ways to add some simple network rendering to the basic app... more on that at a later point

steve

It was said in a thread at reduser that future version of metafuze would work with Pipeline FX and Cube for render farm support. I incorectly interpreted "future versions" as v2 I think. I also read it like Avid would provide modules for those applications but maybe I should had read it like "since it's command line based you can drive it from any command line based distributed renderer out there"...?

Don't get me wrong now. I really dig MetaFuze and so and I am fairly certain that we will move from RedRushes to MetaFuze at our place now. We we very close to do that back in the 1.2 days but there was just to many bugs and limitations in 1.2 but all those issues seems to have been solved and taken care of now. But... At the moment, at least for me. MetaFuze feels like something that has great potential but is more a proof of concept or something. It's really not there yet. Let me explain a bit further what I mean with that. I know it's hard to really express what you mean on forums, so don't get me wrong or get offended or so. =) I sure as heck don't mean to offend anyone with what I write. I just sing out straight from my heart. ;)

The GUI feels a bit clumsy and unpolished at the moment. Sure, there are aspects with it that I love like the customization of panels and columns and so. But it's still a tad rough.

And the "clip based" workflow isn't maybe the best. Maybe it should be "transcode" based so you add a transcode and then the clips to the transcode, insted of the other way around as today. Something like Sorensson Squeeze. To me that would be more logical since you most probably are working on the same production when you transcode and you would want the same settings for everything you transcode. Sure... It works as it is today to but it ain't that intuative and "fail safe".

And from what I can see now you can't save and load transcode settings? Other then with the XML files. That could need an extension so you can build up a library of transcode templates that you can easily access from with in the transcode detail panel.

Would also be sweet with more options to the burn in. So we can burn in any meta data and custom texts and format it a bit more creativly then today.

An other thing that would be nice would be if it didn't lock the gui once it started to transcode. But there instead is an over all progress bar on the total batch and a progress bar for each clip. And that you wile it transcodes can add more clips to it. I keep referring to Sorensson Squeeze, wich I really hate for other reason, but still that GUI has some goodies to eye on.

But the biggest biggie of them all... Full command line support without the XML-files. The XML files means that we in some stage have to use the GUI. If we don't make our own tool to generate the XMLs but that's just cumbersome. =/ If we could access metafuzes all functions from commandline it would be really easy for us to write modules for distributed renders like Smedge or Deadline. I would like to be able to do something like:

metafuze.exe -v -r -d Y:\PROJECT\RED\source\ -b tc,key -c d40 -C 3 -d low -R hg -o Y:\PROJECT\RED\transcoded\

-v verbose
-d directory
-o output dir
-O output dir, create dir for each clip
-r recursivly
-b burn in
-c compression
-C conversion mode
-d debayer
-r Resolution
-f read flags from file

Of course it should also be able to read all those flags from a file so I could call it with something like:

metafuze.exe -v -f Y:\PROJECT\RED\mfuzesettings.txt -d Y:\PROJECT\RED\source -o Y:\PROJECT\RED\transcoded\

All flags on the command line would override what's in the file we called. The file would in this case be a simple line seperated txt and look something like:

-v
-r
-b tc,key
-c d40
-C 3
-d low
-R hg

But I don't know... It took quite a while to get 2.0 out. 7-8 month or something? So it doesn't feel like this product is something that Avid really puts the manhours behind. That it's more someones pet project on the side that has Avids blessings. Nothing wrong with that at all, if that's the case I guess it's more impressive. But it's also sad if that's the case because mfuze has so much potential and we would gladly pay for it if it got the attention it needs. Or bundle it with the other Avid Software and also sell it seperately and sell render node licenses without the GUI cheaper then the GUI ones. But with that said I guess we're looking at another 6-8 months for the next release? =/

Hi Henrik,

Don't worry about offending us we want to hear from real users what they would like and thanks for taking the time to put this comprehensive post together. We are all passionate about what we do and it is via threads like this that we can make a better solution...

Not sure who said what on Reduser but where we are right now is we are developing the building blocks...

The dilemma with MF like any other app in its early days is that we wanted to make it fairly simple (yes I know there is waaay more we can and need to doand your points are well noted, thanks) yet allow it to be used in an enterprise type environment and also by those who can script etc... so given all of these criteria we have landed somewhere in the middle with this version...

You will be happy to learn that many of the things you have pointed out and asked for above are in our longer term plan... As a publilcly traded company I can't say when we will deliver them or which ones however it is threads like this that help us prioritize what comes next...

Regarding the timeframe there were point releases along the way from 1 to 2 and we are already working on the next release based on feedback from the 2.0 version... right now I cannot commit to a timeframe

Over the next few weeks I hope that this forum helps to guide where we take the product and continues to provide feedback and suggestions...

steve

Hey Steve!

Glad you took it the right way. =) And I am really pleased to hear that what I "want" is on the map. I don't know much about programming and application development. But I would have thought that making it work just as a commandline app first and then build a GUI that just uses that commandline.exe would be the easiest and smartest for an application like this. But I might be wrong. But hey... On the other hand maybe this is the case just that all the commands arent available to the user in the current releases?

If I could take a pick of what to put the effort on it would be the command line part. Then we can use mfuze on the farm and from where I'm standing that would put it in a unique spot since there's no other solution for R3D's that is farmable. Sure... There is the problem with quicktimes and farming. I think only apple "have solved" it so they can have several computers render to the same QT. But it would for sure be valid just to send it in chunks of clips instead of chunks of frames. So each node works on one clip at a time. That would make life easier I guess. =)

How activly is metafuze developed? Is there a dedicated team that works with it on a daily basis or is it worked at when "theres nothing else to do"? Maybe an odd question, just being curious. =)

Cheers man

Hi Henrik,

What I try to do is separate the feature set from the implementation and talk about how functionality gets "revealed" to the user...

As you can imagine we have lots of requests for formats, workflows and features from a wide diversity of users... so the xml path was appealing...

In a number of cases I actually don't use the UI at all... an XML is either generated or supplied by the original creating app/service... as an example either as a post process from a render manager for 3D systems or after the scanner has completed populating the dpx files or a copy has completed... in cases like this MF needs to tie into the "business process logic"/ workflow...

As you point out there are other hurdles to some of the stream based essence and this is indeed an area which needs work...

As for how actively we are developing MF. It is in active development, there is a team and we are working on the next release...

Keep the feedback, suggestions and comments coming...

steve

Hi All

i'm getting good render results using Globalstor RedE Rack system.

I'm working on Assimilate Scratch for grading R3D footage.

my spec are:

2x Quad Core 3.2 Ghz overclocked to 4.1 Ghz.

4GB RAM

24 disks Raid configre as Raid 5. (Total of 19 TB from 24x 1TB disks)

Win Server 2003 32bit

i just finish render 1 hour and 33 minutes of R3D in 1 hour and 42 minutes.

(4K 25 FPS into 1080p/25 DNxHD 40)

Couple of questions/advices:

1. i don't want to use me Scratch system for rendering - is there a way to create a render farm with about 6 PC's for those renders?

2. I think that for next build, what will be best will be to do a "transcode" job and not a "File job".i'm getting RED footage everyday that needs to be renderd, it will be great to load all shots and to transcode it all in one click, instad of using the CTRL button so much for chooseing all the files. (something like "Procoder" instad of "Sorenson"). a "save transcode preset"  and a "load transcode preset" is also necessary. 

3.it will great if you can develop a Render Manager - a tool for craeting a priority list for rendering.

4. in my wildest dream i image an email feature that will be able to send me an email while a job has been done.

Thanks for all the good work!

Dori Bashan

Sorry.

I post this theard here by mistake.

i now re-posted it on a new thared on the Metafuze forum and not as a replay to this theard.

please replay on my new post.

Thanks