Latest post Wed, Jun 8 2016 6:48 PM by mmullen. 13 replies.
Page 1 of 1 (14 items)
Sort Posts: Previous Next
  • Mon, May 16 2016 9:24 PM

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 383
    • Points 4,930

    10Gb vs. 1Gb for bin saves

    Greetings,

    Our bins average about 50MB each and our editors usually will have 4 or 5 open at a time. As our projects grow in size we experience very slow bin saves. By slow I mean a spinning beach ball for up to 10 seconds per bin. This is way too long for our workflow when we have a director in the room.  All of our client systems are connected via 1Gb ethernet. We are using a Terrablock system but I hear people of the same issue on an Isis.

     

    Next month we are planning on switching over the Avid Nexis system which gives me the opportunity to redo our switching environment. I've been thinking of moving all edit systems over to 10Gb ethernet with the intention of improving save times. I don't care about bandwidth as we only use DNxHD36 with at most 9 MC seats (at the moment). 

     

    Does anyone know if I should see improved save times with 10Gb vs. 1Gb? I would think so because 10Gb has much faster latency but maybe it's the server or MC software that actually slowing things down and faster networking may not help.

     

    Thanks for any input regarding this.

     

    -Cablet

     

     

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Mon, May 16 2016 9:38 PM In reply to

    • mmullen
    • Top 200 Contributor
    • Joined on Thu, Nov 13 2008
    • Haverhill, MA
    • Posts 568
    • Points 6,500

    Re: 10Gb vs. 1Gb for bin saves

    Are you running your project files off of RAID5 or Mirrored volumes off the Terrablock?

    -Matt M.

  • Mon, May 16 2016 10:23 PM In reply to

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 383
    • Points 4,930

    Re: 10Gb vs. 1Gb for bin saves

    Hi,

    We are using Mirrored volumes for all projects and Raid5 for all media.

    We get very fast throughput with the Terrablock and despite using every video and audio layer we've never had an issue with playback. It's just the bin saving that's killing us.

     

    Thanks.

     

    -Cablet

     

     

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Tue, May 17 2016 4:42 PM In reply to

    • mmullen
    • Top 200 Contributor
    • Joined on Thu, Nov 13 2008
    • Haverhill, MA
    • Posts 568
    • Points 6,500

    Re: 10Gb vs. 1Gb for bin saves

    Does a new project save bins slowly as well?

    -Matt M.

  • Tue, May 17 2016 5:19 PM In reply to

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 383
    • Points 4,930

    Re: 10Gb vs. 1Gb for bin saves

    A new project saves very quickly. But within days it crawls to a slow again. What we currently do to help with this is we rename the Attic file daily so MC creates a new one. This helps greatly. We also disable spotlight at the OS level and disable MC's find feature. Small bins save faster than large bins as one might expect. Our largest bins sit just under 200MB but the average is around 50MB. The project settings files save instantly.

     

    As a test I copied the entire project and media to a USB 2.0 drive (which is slower than 1Gb) and the bins save almost instantly. The drive was formatted to match all of the volumes. 14 Volumes.

     

     

     

     

     

     

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Tue, May 17 2016 6:29 PM In reply to

    • mmullen
    • Top 200 Contributor
    • Joined on Thu, Nov 13 2008
    • Haverhill, MA
    • Posts 568
    • Points 6,500

    Re: 10Gb vs. 1Gb for bin saves

    Can you send me the output of the avid console.

     

    matt@facilis.com

    -Matt M.

  • Tue, May 17 2016 7:59 PM In reply to

    Re: 10Gb vs. 1Gb for bin saves

    Have you tried saving WITHOUT the media online (unmount the media workspaces)?

    Is the saving process faster?

    MC 8.6.1, Windows 10, ASROCK X99M, Intel 5930K, Zotac GTX970, 32GB RAM, Transcend 512GB M.2 SSD & Dell U2713H. MBP 2012 Thunderbolt, OSX 10.10.5, MC... [view my complete system specs]
  • Wed, May 25 2016 10:16 PM In reply to

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 383
    • Points 4,930

    Re: 10Gb vs. 1Gb for bin saves

    I'll give this a shot as soon as I can with one of the large projects. I did hear back from both the Nexis product manager and the Media Composer product manager and they both told me that I most likely will not see any difference with bin saves using 10Gb. The slow bin saving doesn't have much to do with the network (though saving the same bins on a usb drive is instant).

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Thu, May 26 2016 12:36 PM In reply to

    Re: 10Gb vs. 1Gb for bin saves

    I've been using Unity/Isis for 10 years now and slow saving as the project grows has always been apparent.

    When you take the same project and open it from a local drive it feels like you've gone into warp speed compared to the sluggish Isis open/save.

    I'm really surprised that Avid hasn't improved the situation yet.

     

     

    MC 8.8.2 OSX 10.12.3 Quicktime 7.6.6 Nitris DX [view my complete system specs]
  • Thu, May 26 2016 2:51 PM In reply to

    • kkordisch
    • Not Ranked
    • Joined on Mon, Dec 5 2011
    • Posts 85
    • Points 1,085

    Re: 10Gb vs. 1Gb for bin saves

    How big is your project? Have you changed the default of the Search DB folder from the Project to the Local as well?

  • Thu, May 26 2016 5:03 PM In reply to

    • cable
    • Top 500 Contributor
    • Joined on Sun, May 27 2007
    • Pacific NW
    • Posts 383
    • Points 4,930

    Re: 10Gb vs. 1Gb for bin saves

    Our project folder is usually around 25GB by the end of a project. We do set the Search DB to the local system but we disable find anyway so I don't think those databases grow. One thing I have found that helps a lot is making sure we don't have too many Media Folders. I've found through detailed testing that the more media databases there are the longer it takes to do anything. I make sure we have between 4000-5000 files per folder to keep the folder count down. By doing this on our last project we decreases the MC project loading from 90 seconds to about 17 seconds.

     

    I think it's just a fact of life now for network shared projects.

    20 x Media Composer 8.6.5 Mac Pros 32MB RAM OSX 10.11.6 AJA and Nitris DX hardware Avid Nexis Enterprise [view my complete system specs]
  • Mon, Jun 6 2016 10:04 AM In reply to

    Re: 10Gb vs. 1Gb for bin saves

    Try emptying the Unity Attic folder.  This can bloat up in size to enormous proprtions in a big job.   I back the projects up every night and the Attic folder with it and delete the it every few days to speed bin saves up. Also, try the command line "manageama off" if you are not using AMA on your edit machines or manually empty the AMA Metadata folders.

    MC/SN 2018.6+7+8+9+12 x 16 Z400/420/440 PC's on W7 x64. 2 x Mac, 4 x PT 12.8.1 HDX, S6 all on PCs. ISIS 5000 X 2. [view my complete system specs]
  • Mon, Jun 6 2016 7:20 PM In reply to

    • BobRusso
    • Top 50 Contributor
    • Joined on Thu, Oct 13 2005
    • New Jersey
    • Posts 2,243
    • Points 27,700
    • Avid Employee
      Moderator: Avid ISIS
      Moderator: Avid NEXIS PRO
      Moderator: Avid Storage Products

    Re: 10Gb vs. 1Gb for bin saves

    How big is the Attic? Do you have a common workspace for the Projects? This issue can be related to an Attic with a large number of bins in it. I recommend zipping the Attic on the Workspace and then deleting the Attic folder leaving the zipped file. This way if someone needs something they can still get access to it.

    Sometimes one of the bins in the Attic can be corrupt and MC has a difficult time deleting it. Creating a new Attic will resolve this issue. Furthermore, having a large number of bins will use a disproportionate amount of space on Workspace and will add to the overall file count.

    How much media is in the project/workspaces? Do you have workspaces mounted with media that you aren't using? Media Composer is tracking media in every workspace with an Avid MediaFiles folder or linked media.

    Some of the easy things to try is unmount workspaces you aren't using. ISIS/NEXIS supports 1024 workspaces with no limits on the number of times they are resized or a hit on performance during the resize. With NEXIS try to create a larger number of smaller workspaces so media in workspaces you aren't using can be unmounted.

    You can use MediaMover or MDV_AVID to move media out of the Avid MediaFiles folder so Media Composer is tracking less media. It's also a great way to find orphaned media from projects that should have been deleted.

    It is the same for linked media. Even if the media has been transcoded and that's what being used, if the file based media is available, MC is still tracking it. If possible, move the file based media offline.

    Delete unused precomputes; like bins these are small files that use a disproportionate amount of storage.

    Having said all of this, there is a known issue when the Media Composer Project file gets large and the save times increase. As a test, create a new project and copy the bins to it. Does it open and save quickly?

     

    https://www.youtube.com/avidevangelistbob [view my complete system specs]

    -- Bob Russo Post-Production Workflow Manager - NFL Films

    Bob Russo on Expert Connections

  • Wed, Jun 8 2016 6:48 PM In reply to

    • mmullen
    • Top 200 Contributor
    • Joined on Thu, Nov 13 2008
    • Haverhill, MA
    • Posts 568
    • Points 6,500

    Re: 10Gb vs. 1Gb for bin saves

    I've also seen renaming the ManagedAMAFolders.xml file in Users/Public Documents/Avid Media Composer/AMA Management make a big difference on the windows side.

    -Matt M.

Page 1 of 1 (14 items)

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller