Latest post Tue, Feb 21 2017 10:49 AM by NickScale. 14 replies.
Page 1 of 1 (15 items)
Sort Posts: Previous Next
  • Tue, Feb 7 2017 9:49 AM

    • NickScale
    • Not Ranked
    • Joined on Thu, Apr 25 2013
    • Posts 22
    • Points 240

    Facility tells me AMC will slow down with large projects

    I'm cutting a TV show at a large post house in London who tell me it is standard procedure to split the edits of different episodes into separate Avid projects. They tell me that a single project containing eight 1-hour episodes will get "slow and messy".

    I'm confused. It's not their problem if the project gets messy, that's up to my editorial team keeping the project organised.

    Also I don't understand how a project would get slower if it has a lot of bins and sequences in it, because we would still only open the bins and sequences we are working on, so what does it matter if they are in one project or many projects?

    The rushes are on a single ISIS and the bins are manually replicated across every project. There is no automated way that the bins remain mirrored. They have suggested creating an additional Generics project for music/SFX/GFX, but this means we must switch to it every time we want to ingest something. This seems onerous and inconvenient.

    Can someone explain to me why a large project might get slow?

    This might be an ISIS related issue, rather than MC. The clients are typically HP Z420 Xeon 3.2GHz 16GB Win 10 machines with MC 8.6.4.

    Thanks

    Nick

  • Tue, Feb 7 2017 2:57 PM In reply to

    • jef
    • Top 50 Contributor
    • Joined on Sun, Feb 26 2006
    • Maryland
    • Posts 3,746
    • Points 44,930

    Re: Facility tells me AMC will slow down with large projects

    I do not work on ISIS. (We use Facilis TERRABLOCK - similar).

    I would agree with the advice you have been given for many reasons AND from an editors perspective.  And I would not even put "the project getting slower" as one of the reasons.  So I may not be really addressing your question, but I am speaking to the larger issue.

    Mainly it would be about simple organization and safety.  No matter how well organized your team is, human nature being what it is and the pressures of editing being what they are, one big project is asking for problems.  Having a project per episode plus a template for how each project will be organized will help all the editors.  I know you can impose a project structure on the one big project, but mistakes happen.  Stuff for Ep3 gets placed in the Ep5 folder and then the editor tasked with making a "quick change" can not find the material needed.  I have been there too many times.

    Safety - well if something get corrupted in the one big project you are only as safe as your last project backup.  If that was last night and you are at the end of the next day, well, a lot can be lost.  At least in some situations.

    Avid makes sharing stuff from multiple project quite easy. And I don't find switching projects to ingest something that bad. Especially given the benefits of having only one place that music/SFX/GFX will be stored.

    This is all just my opinion formed from a few years of experience.

    Jef

    Avid DS 11.0.2 R.I.P | MC "Well, it depends ..." mostly 8.12.9|OS 10.14.x - iMac Pro 2019 - home system MacPro Dual 2.8 8core GTX680 "Harpertown"... [view my complete system specs]

    _____________________________________________

    Jef Huey

    Senior Editor

    Old Stuff  http://vimeo.com/album/3037796

  • Wed, Feb 8 2017 1:27 AM In reply to

    • jwrl
    • Top 25 Contributor
    • Joined on Thu, Oct 13 2005
    • Melbourne, Australia
    • Posts 8,420
    • Points 97,425

    Re: Facility tells me AMC will slow down with large projects

    Totally agree with jef's post.  The argument is not whether Avid will slow down or not, but whether it's smart to do at all.  I have always worked with episode projects, then stock, then any other specialised project needs for the production.  And to ensure that anything shot for a particualr episode that subsequently becomes stock is handled correctly, that footage is opened in the stock project and consolidated.  That attaches the new version to the stock project, and protects it from being accidentally deleted when the individual episodes are cleaned up.

    MC 7.0.4 - Asus P6T Deluxe V2 mobo - Intel i7 920 2.66GHz - Windows 7 Ult64 SP1 - nVidia Quadro FX 1800 - 16 Gbyte low latency DDR3 RAM - Internal 8 Tb... [view my complete system specs]
  • Thu, Feb 9 2017 8:12 AM In reply to

    • luca.mg
    • Top 25 Contributor
    • Joined on Thu, Oct 13 2005
    • Roma - Italy
    • Posts 5,737
    • Points 67,310

    Re: Facility tells me AMC will slow down with large projects

    Totally agree with jef and jwrl, and want to top it all with media management: when an episode is completed it's straight forward to get rid of the unused shots and precomputes.

    Symphony 20.9, BM Intensity Pro 4k, Windows 10, i7-5930K, 32 GB ram, Quadro K620 [view my complete system specs]

    peace luca

  • Thu, Feb 9 2017 11:20 AM In reply to

    • NickScale
    • Not Ranked
    • Joined on Thu, Apr 25 2013
    • Posts 22
    • Points 240

    Re: Facility tells me AMC will slow down with large projects

    I appreciate all the comments, but just to reiterate the facility said the "project will slow down". I don't believe they are correct and I'm looking for someone to broaden my mind about why this might occur.

    I agree with some of the things you are all saying about media management and safety, but we are only across three suites cutting an 8-hour 8-ep season. We will definitely not be reorganising, consolidating or getting rid of any assets mid-season. The risk of human error in replicating dozens of bins, timelines, markers, and other assets across nine different projects is considerable, and the loss of productivity in repeatedly switching projects to ingest generics is not practical.

    But bottom line: this was not the reason given by engineering so that is not my concern in this thread.

    Nick

  • Thu, Feb 9 2017 12:55 PM In reply to

    Re: Facility tells me AMC will slow down with large projects

    The facility is telling you the truth. But the factors that play a role are several and structuring your workflow so you are able to minimize the impact is adviced.

    First of all avoid using a large amount of AMA links in multiple bins. This slows down MC big time with ISIS shared projects. I'm hearing from clients that 8.6.5 is performing better in combination with shared storage but have not been able to confirm myself.

    The second reason projects slow dow is the Attic. In large projects it is adviced to move all bins manually out of the Attic to another backup location. This can give drastic performance increases.

    The reason for the slow down when using shared storage is not officially known, at least I have not seen any info from Avid. The only knowledgebase article that I know is this one: http://avid.force.com/pkb/articles/en_US/troubleshooting/Ways-to-speed-up-Media-Composer-bin-save-load-switching-between-applications-and-startup 

    The current workaround it to use multiple projects, for example 1 per episode and also setup a project with all graphics, music etc... this media should not be ama linked but classic Avid media, either imported/consolidated/transcoded. Then use 'open bin' and manually open bins from other projects as needed.

    Hope this helps

     

     

     

    From the old Apple Quadro 950 to HP Z8xx. My current own system: 1x Z420 E5 1650 16GB memory, 128GB SSD, 1x XW8600, 2x 3.0Ghz Quadcore, 12GB memory... [view my complete system specs]

    Jeroen van Eekeres 

    Technical director, Broadcast support engineer, Avid ACSR.

     

    Always have a backup of your projects....Always!!!! Yes Always!!!!

    A.V.I.D....... Another Version In Development

    www.mediaoffline.com

     

     

     

  • Thu, Feb 9 2017 1:29 PM In reply to

    • luca.mg
    • Top 25 Contributor
    • Joined on Thu, Oct 13 2005
    • Roma - Italy
    • Posts 5,737
    • Points 67,310

    Re: Facility tells me AMC will slow down with large projects

    Unnecessarily complex, or complicate, naturally lends itself to chaos, being It slowlyness or else. I think that the facility claim is based on past experience, and to be able to make an opinion I'd need more info about the planned workflow, eg resolution, multicamera or not, effects intensive, footage amount, shared storage platform and the kind of network in use, the editing workstations. I've cut a seven episodes docudrama, shot with two to three cameras with separate audio, the First season has been in a single huge project, with graphics and music in their own projects, iMacs 5k, DNxHD 120, MC 6ish and ISIS 5000, painfully slowl, increasingly in time. Second season DNxHD 85 with each episode, the graphics, and the music, in their own project, this was more manageable, but to be fully honest the following season went in DNxHD 36 and online at picture lock. Large projects like this need very good assistant editors and a little wrangling science.

    Symphony 20.9, BM Intensity Pro 4k, Windows 10, i7-5930K, 32 GB ram, Quadro K620 [view my complete system specs]

    peace luca

  • Thu, Feb 9 2017 1:41 PM In reply to

    • NickScale
    • Not Ranked
    • Joined on Thu, Apr 25 2013
    • Posts 22
    • Points 240

    Re: Facility tells me AMC will slow down with large projects

    Thank you Jeroen, this is really useful info.

    We have eliminated all linked media since we started editing so that shouldn't be a problem.

    The Attic however could be an issue. Avid has been slow at saving bins since the start.

    We have Bin autosaves set to a maximum of 1000 attic files and 50 versions per bin. Is this normal/high?

    I will ask facilities to check if the Unity attic is getting bloated at the end of each week. Is it possible/advisable to use a local Attic rather than Unity Attic?

    Nick

  • Thu, Feb 9 2017 2:13 PM In reply to

    Re: Facility tells me AMC will slow down with large projects

    Either way would work but the seperate project per ep method puts the managment into the hands of the facility who will 100% do a far superior job than a bunch of freelancing ediors ;)

    On the other hand, running all eps in a single project is not an issue in my view. I've ran projects this way across Unity shared storage, Isis and our current Nexis. 

    Slowing down much? Don't edit from AMA - ingest machines purge AMA management folders regularly and don't let the attic fill over 10GB and you'll be fine.

    If all media is managed / ingested and metadata recorded at source, then you can identify it all regardless of which project it's in. 

    Keep an eye on the total bandwidth of all machines via your management console - generally more edits means more bandwidth usage. Obviously ingest codec choice is a factor too.

  • Fri, Feb 10 2017 9:08 AM In reply to

    • NickScale
    • Not Ranked
    • Joined on Thu, Apr 25 2013
    • Posts 22
    • Points 240

    Re: Facility tells me AMC will slow down with large projects

    luca.mg:

    I'd need more info about the planned workflow, eg resolution, multicamera or not, effects intensive, footage amount, shared storage platform and the kind of network in use.

    We're offlining in 25i PAL at 20:1. It's multi-camera, either 2 or 5 per setup. I don't know details of the ISIS, but bandwidth is fine for 5 concurrent SD streams. The workstations are HP Z420 3.2GHz Xeons with Nitris DX. Maybe 40 hours of rushes per ep. Pre-rendered GFX with alpha, no other VFX.

    We have dedicated edit support who handle rushes ingest and conform. Their default approach is to use episode projects regardless of the production requirements.

    I've worked on shows with 24 editors working on 16x1h episodes in a single AMC project. I think this current show is well within a single AMC project's capabilities (though admittedly I don't know the ISIS/network specs).

    Edit: also we're not cutting any linked media, it's all transcoded.

    Nick

  • Fri, Feb 10 2017 12:09 PM In reply to

    • luca.mg
    • Top 25 Contributor
    • Joined on Thu, Oct 13 2005
    • Roma - Italy
    • Posts 5,737
    • Points 67,310

    Re: Facility tells me AMC will slow down with large projects

    20:1 SD? No comment. Seems like an "ordinary job", which could be handled in a single project, being this the most sensibile decision or not, and it seems that this is the realm of personal opinions.

    Symphony 20.9, BM Intensity Pro 4k, Windows 10, i7-5930K, 32 GB ram, Quadro K620 [view my complete system specs]

    peace luca

  • Fri, Feb 10 2017 1:42 PM In reply to

    • NickScale
    • Not Ranked
    • Joined on Thu, Apr 25 2013
    • Posts 22
    • Points 240

    Re: Facility tells me AMC will slow down with large projects

    luca.mg:

    20:1 SD?

    Apologies, it's 10:1 but still pretty low.

    Nick

  • Thu, Feb 16 2017 9:45 PM In reply to

    Re: Facility tells me AMC will slow down with large projects

    Do what they say.  They will have constant and evolving experience of how to best manage projects which you do not have.  They will have anecdotal evidence based on previous experience that may not be quantifiable but that has proven to be true over time.

    It’s likely that you won’t be ingesting anything yourself anyway as that is what the MCR assistants are for so moving between projects to digitise shouldn’t be a problem for you.  It’s very bad form to let clients ingest their own material.  This can be a nightmare for conforms when every editor has their own labelling system or doesn’t have one at all, or imports all the gfx and v/o to the local C drive.

    Having lots of bins in the project will cause the project to grow in size which will mean your project will take ages to open.

    There is minimal risk of human error when replicating bins across projects.  The assistants in these post houses do it all the time and know how to deal with this.  There is a risk however, of one editor accidentally dragging someone else’s bin in to some folder somewhere and then causing havoc when no one can find their bins.  I’ve seen this happen a million times. Separate projects will minimise editor on editor collisions!

    You don’t need to open a project to use a bin within that project, it’s as simple as ctrl+o and open the bin directly in your project, it will then be available at the bottom of your project in the ISIS Bins folder that is created when you open a bin from another project.

    At the end of the day, why resist? The post house has a tonne of other projects going on and by asking for things to be done differently for your project you are actually more likely to cause yourself extra problems if not everyone is on the same page regarding project organisation.

    And yes, my number one concern would be that your project corrupts and every editor has to stop working whilst the projects are restored from backups.  If you have separate projects and one corrupts, at least the other editors can continue. 

     

    As far as I can see it’s no skin off your nose to organise by episode, your editors will thank you for the added clarity and the post house gets to have peace of mind that project size is not a problem when they inevitably have to troubleshoot some problem. Go with the flow man.  

    HP ZBook G6 15 2.6Ghz 6-core i7 9th Gen Nvidia Quadro T2000 4GB 64GB RAM Win 10 Pro MC Ultimate 2018.12.11 MacbookPro 15 2019 6 core 2.6GHz i9... [view my complete system specs]
  • Sun, Feb 19 2017 10:31 AM In reply to

    • Jojo Likar
    • Not Ranked
    • Joined on Sat, Nov 26 2011
    • Austria
    • Posts 60
    • Points 715

    Re: Facility tells me AMC will slow down with large projects

    Just wanted to share my experience on this topic.

    I have never had the experience of a slow system because of project size. I work part time in a shared storage enivronment (ISIS 5500) with three clients. We have some big projects that contain weekly half-hour shows for one year (eg. [ShowName]_[Year]) However these are mostly fnished shows for audio dubbing, smaller editorial changes or reediting of certain parts. When we have a new show edited from raw footage we have a seperate project. These big yearly project files however open/save/respond as fast as small projects. (ann. all material is imported, nothing AMA linked, everthing HD)

     

    On my own system I'm editing a 10x20minutes series at the moment ( ~200 bins, ~ 75 hours footage, 4K DNxHR LB) and i have also not experienced a slowing down of the system.

     

    One of the reasons why I like to work with Media Composer is because I'm able to create big projects like this without having to fear of instability. And most important the fact that project file and bins are seperate files.

    That's just my experience but often it makes sense to split big projects into smaller parts for organisation. When I set up a bigger project with possibly multiple shows or different versions in the end I always ask myself to what project I would like the media files related to. (for organisation in the media tool) If it makes sense to split up the project or if it's easier to have it all in one place and not risking to delete the media files from one project whereas I was still using them in another project.

     

    But of course when you have to collaborate or fit into an existing ecosystem then it's important to keep everyone happy.

    Media Composer 2018.2 | Win 7 64-bit | BMD Decklink Mini Monitor 4K | Intel i7-6800K | X99-EWS | 64 GB RAM | GTX 970 | Artist Control & Transport ... [view my complete system specs]
  • Tue, Feb 21 2017 10:49 AM In reply to

    • NickScale
    • Not Ranked
    • Joined on Thu, Apr 25 2013
    • Posts 22
    • Points 240

    Re: Facility tells me AMC will slow down with large projects

    ripvanmarlowe:
    Do what they say.  They will have constant and evolving experience of how to best manage projects which you do not have.

    They are a large facility with non-technical producers who implement cookie-cutter processes for every production. I don't think their approach was good in this case.

    ripvanmarlowe:
    It’s very bad form to let clients ingest their own material. 

    Rushes - yes. Music/SFX/GFX - no. Asking MCR every time we want to ingest a WAV is not practical.

    ripvanmarlowe:
    Having lots of bins in the project will cause the project to grow in size which will mean your project will take ages to open.

    The additional seconds required to open a larger project are insignificant compared to Avid's usual sluggishness saving large bins and cutting long timelines. Neither of these issues is due to project size.

    ripvanmarlowe:
    At the end of the day, why resist?

    Because we were already cutting the third of eight episodes when they told us they had set up episode projects. It would have been disruptive and risky to shift to their methodology mid-run. None of us wanted episode projects.

    ripvanmarlowe:
    my number one concern would be that your project corrupts and every editor has to stop working whilst the projects are restored from backups

    The way we're working is: three suites block cut the stories for an ep, then one suite stitches and fine cuts it. With episode-based projects, project corruption could still suspend 3 out of 4 suites. So yes, there is a margin of security that one suite could keep running, but I don't think that's proportionate to the overheads in maintaining 9 projects.

    I personally have never experienced a corrupt project (except across different AMC versions, not pertinent in this case). The many corrupt bins I've had over the years could be recovered in seconds. But, to reiterate: my question was not about security or redundancy.

    So far no one has provided compelling reasons why this might cause an unresolvable slowdown, other than bloated attics (easily managed) or excessive linked media (we have none). We're now in week 7 and there is no noticeable slowdown.

    Nick

Page 1 of 1 (15 items)

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller