Latest post Sun, Nov 18 2018 11:46 PM by Dom Q. Silverio. 10 replies.
Page 1 of 1 (11 items)
Sort Posts: Previous Next
  • Fri, Mar 23 2018 4:13 PM

    • mnbkyle
    • Not Ranked
    • Joined on Tue, Sep 15 2015
    • Minneapolis, MN
    • Posts 29
    • Points 340

    ISIS 5500 Disk Performance Degraded

    Hey all - I'm geting Disk Performance Degraded warnings on our ISIS 5500. It keeps referring to DISK 1. I'd like to replace the ISB but none of the LED indicator lights are red so I'm not sure which one to replace.

    A couple questions:

    1) Is DISK 1 referring to the ISB in the very top left of the 16 ISB's?

    2) What is the procedure for replacing? Simply pull out the defective ISB and insert the spare? Do I need to shut down the ISIS before replacing or leave everything running?

    See System Logs:

    Filed under: , , , , ,
  • Sat, Mar 24 2018 12:17 PM In reply to

    Re: ISIS 5500 Disk Performance Degraded

    Hi Kyle.

    The disk 1 is related to 1 of the 3 raid sets inside the isis. Not to an ISB with a microcontroller with its own IP as found in isis 7000.

    What you need to do is to open the LSI (now called Avago) megaraid storage manager utility on the isis. It should be installed by default but you can also dowload it from Avid here:

    http://avid.force.com/pkb/articles/en_US/faq/MegaRaid-software-for-the-following-products 

    In there you will find all 16 drives and you need to find if and how many drives show an elevated media error count.

    Healthy drives show 0 and from experience seeing a drive with less than 10 is not worrying. When you see the degration you mention, you will find 1 or more drives with a much higher media error count, like >100 or even >1000. Although not failed yet you should replace these drives with new drives from Avid.

     

    From the old Apple Quadro 950 to HP Z8xx. My current own system: 1x Z420 E5 1650 16GB memory, 128GB SSD, 1x XW8600, 2x 3.0Ghz Quadcore, 12GB memory... [view my complete system specs]

    Jeroen van Eekeres 

    Technical director, Broadcast support engineer, Avid ACSR.

     

    Always have a backup of your projects....Always!!!! Yes Always!!!!

    A.V.I.D....... Another Version In Development

    www.mediaoffline.com

     

     

     

  • Mon, Mar 26 2018 3:18 PM In reply to

    • mnbkyle
    • Not Ranked
    • Joined on Tue, Sep 15 2015
    • Minneapolis, MN
    • Posts 29
    • Points 340

    Re: ISIS 5500 Disk Performance Degraded

    Thanks for the reply! I've installed the MSM software and have identified Slot 8 as the failing driving with a media error count of 193.

    I know ISIS 5500 is mostly an automated process - would you recommend waiting for the ISB to fail, and let the ISIS redistrubute to the spare, then replace? or swap it out before then? And if swapping immediately - can this be done while the ISIS is running?

    I have a new spare ready, I'm just trying to get this process firmly understood before pulling out the ISB.

  • Mon, Mar 26 2018 5:22 PM In reply to

    Re: ISIS 5500 Disk Performance Degraded

    I would only force failover if it is actually causing you issues. Often "network degraded" can be safely ignored.

     

    If users are having performance or stability issues (bin not saving, crashing, etc.) then you can pull the dying drive and force drive recovery to the hot spare.

    Make sure you confirm it is the correct drive. Check the serial number before you pull it.

     

     

     

     

    MC 8.6.1, Windows 10, ASROCK X99M, Intel 5930K, Zotac GTX970, 32GB RAM, Transcend 512GB M.2 SSD & Dell U2713H. MBP 2012 Thunderbolt, OSX 10.10.5, MC... [view my complete system specs]
  • Fri, Mar 30 2018 3:26 PM In reply to

    • mnbkyle
    • Not Ranked
    • Joined on Tue, Sep 15 2015
    • Minneapolis, MN
    • Posts 29
    • Points 340

    Re: ISIS 5500 Disk Performance Degraded

    Thanks Dom,

    We weren't experiencing performance issues until just today. I'm trying to transcode AMA linked XAVC FS7 Media, a common procedure, and I keep getting the following error message:

    Exception: DIDReader::XraiseUnexpectedStreamStatus async file system error.

    This also happens when trying to transcode AMA linked XDCAMEX media.

    Do you think this is a sign to force failover?

  • Sat, Mar 31 2018 9:45 PM In reply to

    • mnbkyle
    • Not Ranked
    • Joined on Tue, Sep 15 2015
    • Minneapolis, MN
    • Posts 29
    • Points 340

    Re: ISIS 5500 Disk Performance Degraded

    SOLVED: I've replaced the degraded ISB. What follows are the steps I took after the recommendations in this thread:

    -downloaded MegaRaid Storage manager onto the ISIS. Logged into the IP to display the slot configuration.

    -located slot 8 which listed a media error count of over 500. All others had a count of zero.

    -right clicked on slot 8 - select locate - One of the 16 ISB's began blinking a single red indicator light. Slot 8 ISB confirmed. Took photo of slot 8 ISB SN displayed by MSM.

    -Disconnected all workstations from ISIS

    -Pulled ISB in slot 8 forcing failover. Confirmed SN on physical drive with photo taken. Confirmed, pulled the correct ISB.

    -MSM displays the hotspare ISB turned into an active ISB

    -Insert Cold Spare ISB into empty slot.

    -ISIS management console confirms drive is rebuilding, slowly but surely.

     

    Here are some performance issues that led me to force failover:

    *async file system error
    *failed to get sample size from ama plugin
    *unable to transcode ama linked media
    *unable to complete an MXF OP1a sequence export (no error message displayed, just failed to export a file)

    Since replacing the ISB these errors have stopped and we're back to normal :)

     

     

  • Tue, Apr 3 2018 2:32 PM In reply to

    Re: ISIS 5500 Disk Performance Degraded

    Well done!

    Most of the MC/NC, Interplay, Nexis, ISIS, Unity stuff. [view my complete system specs]

    Avid reseller ACSR at Telmaco

    http://www.telmaco.gr/en/

  • Sat, Nov 17 2018 10:42 PM In reply to

    • doug shannon
    • Not Ranked
    • Joined on Sat, Nov 17 2018
    • London
    • Posts 6
    • Points 100

    Re: ISIS 5500 Disk Performance Degraded

    One of my biggest bugbears of several with Isis, 5500 in our case, is it’s singular inability to realise when a disk is dying and boot it out of the raidset. Instead we have to rely on annoyed editors complaining about an inability to transcode or write files to a workspace and then get out my Sherlock Holmes hat and magnifying glass to find out which of 144 drives has failed this time. Since we have a drive failure roughly every 4 weeks I get to do this a lot.

    Frankly it’s utterly amateurish regardless of whether the fault lies with LSI or Avid, and is one of the reasons our next storage system is unlikely to have Avid on the bezels.

  • Sun, Nov 18 2018 10:49 AM In reply to

    Re: ISIS 5500 Disk Performance Degraded

    doug shannon:
    get out my Sherlock Holmes hat and magnifying glass to find out which of 144 drives has failed this time.

    100% agree that LSI+Avid are letting the system down with bad design. I've been searching if the LSI card/driver could use a media error count amount to initiate the hot spare replacement but never found anything. Sadly Avid also did not find it necessary to create the ability to monitor individual drives, initiate a replacement or automate replacement in the ISIS web page (through the MegaCLI or StorCLI). As ISIS 5000 was already end of development and will go end of support in a month, I don't think our criticism will serve any purpose.

    doug shannon:
    one of the reasons our next storage system is unlikely to have Avid on the bezels.

    The good news is that Avid corrected this in Nexis. Nexis always provided monitoring of all individual media drives from day 1 and since 2018.4 also the internal SSD drives for the SDA. I consider Nexis a great step forward combining features from Unity, ISIS5x00 and 7x00. And at it's current price I believe you should keep Avid storage on your options list.

    That said, as nothing is perfect, I still believe Nexis can be improved especially when it comes to nic teaming and how client and network issues can affect overal system stability. In my opinion Nexis client needs more enhanced (setup) fault analysis and error reporting as very similar Sherlock Holmes methods are needed when for no obvious reason systems can't write to workspaces or not write with the correct speed al of a sudden. 

     

     

    Most of the MC/NC, Interplay, Nexis, ISIS, Unity stuff. [view my complete system specs]

    Avid reseller ACSR at Telmaco

    http://www.telmaco.gr/en/

  • Sun, Nov 18 2018 11:35 PM In reply to

    Re: ISIS 5500 Disk Performance Degraded

    I have not seen a product in the Video SAN/NAS space that proactively "fails" the drive in order to initiate an early rebuild. I have tried Terrablock, EditShare, XSAN/StorNext, Promax, etc...

    With ISIS Toolbox SMART feedback we usually can figure out which drive is about to fail.

    With Nexis, Avid does have a drive state called "dying". But we have not experienced it other than through a bug so I can't tell you how well it will work.

     

     

     

    MC 8.6.1, Windows 10, ASROCK X99M, Intel 5930K, Zotac GTX970, 32GB RAM, Transcend 512GB M.2 SSD & Dell U2713H. MBP 2012 Thunderbolt, OSX 10.10.5, MC... [view my complete system specs]
  • Sun, Nov 18 2018 11:46 PM In reply to

    Re: ISIS 5500 Disk Performance Degraded

    IMHO, Avid ISIS is by far the best video shared storage solution we have ever experienced.

    It is the easiest to deploy. Easiest to administer. Definitely easiest to maintain. 

    There are storages that are cheaper and faster. But ISIS overall is still the best.

    MC 8.6.1, Windows 10, ASROCK X99M, Intel 5930K, Zotac GTX970, 32GB RAM, Transcend 512GB M.2 SSD & Dell U2713H. MBP 2012 Thunderbolt, OSX 10.10.5, MC... [view my complete system specs]
Page 1 of 1 (11 items)

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller