Friday, December 9, 2016

[Avid-L2] Re: OTish: Software for running Checksum on Data Copies Suggestions

 

We use TeraCopy on all our Win machines. 
It replaces Win's own copying function, so it gives you the check automatically - and a list of files if you want that. 

Keld 
FilmSolutions

__._,_.___

Posted by: Keld von Eyben <keld@voneyben.dk>
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (1)

Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.

this is the Avid-L2

.

__,_._,___

Thursday, December 8, 2016

Re: [Avid-L2] Drop Frame DPX?

 

One of Avid's DPX export options is to base the frame count on time code.  So if I export with that option a 1:00:00:00 start time code becomes 86400.  With one second pad at the top starting at 59:59:00 the resulting first DPX frame is numbered 86376 which is 24 frames before 1:00:00:00.  This correlation is useful if I have to make a patch fix and the facilities making the IMF files from the DPX seem to prefer it this way.

Perhaps when dealing with individual shots for VFX etc... the frame count is more helpful but at least in my case of delivering a final master sequence basing it on the sequence time code has worked well.

If I get some research time I'll see what would happen in a 29.97 project exporting DPX from a drop frame sequence.  I would think the resulting time code based option would yield a 2 frame skip at the even minutes except for the 10's of minutes.  Just a curiosity at this point but I remember being burned when people would make 23.976 dubs adding pulldown for offline with drop frame which messed up the online conform later so I figure it's only a matter of time before I get a DPX with a similar issue.



---In Avid-L2@yahoogroups.com, <jeff@...> wrote :

You can think of DPX as just an image sequence. Each frame increases its number sequentially, never skipping a number. Those filenames don't relate back to timecode. Just the order of every frame. No matter what framerate the project is, the DPX files will all be labeled sequentially (no break in numbers)

If you were to take these files in to something like After Effects, After Effects wouldn't have the ability to tell what the framerate of the sequence is. You'd have to tell it yourself. From there it could put timecode on it for you.

This is one of the reasons why when you talk w/ Nuke or Flame or 3D artists they will often talk about frame number rather than a timecode of the area they are working on.

This used to come up more when we were cutting SD in 29.97, but sourcing from 24fps film... Flame would need to remove the pulldown to work on shots... so now it's got a new framerate and new timecode... anyways, I often remember discussions similar to: "... on the fifth frame of that shot..." rather than: "...on 02:15:12:10..." 

Jeff

------------------
Jeff Hedberg

Director of Operations
Union Editorial
575 Broadway,6th floor
New York, NY 10012

On Dec 7, 2016, at 7:37 PM, John Moore bigfish@... [Avid-L2] <Avid-L2@yahoogroups.com> wrote:


With all the fun things I'm learning about 4K and DPX exports from Avid I just had a thought about Drop Frame in a DPX world.  So far all my 4K is 23.976 so there is no Drop Frame issues.  I export basing the frame count on sequence time code which is very helpful when making patch fixes etc...  

I haven't tried this but does anyone know if I was working in a 4K project with 29.97 frame rate where drop frame was used.  If I export a DPX with frame count based on time code what happens at the drop frames?  Would the break in sequential numbers cause the DPX to burp?  I know it is possible to just have the frame count ascend and ignore time code but where's the fun in that.  Basing the frame count on time code is really helpful.  Anybody know if a DPX with frame count based on drop frame time code would work correctly?
 
John Moore Barking Trout Productions Studio City, CA bigfish@...


__._,_.___

Posted by: bigfish@pacbell.net
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (3)

Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.

this is the Avid-L2

.

__,_._,___

Re: [Avid-L2] OTish: Software for running Checksum on Data Copies Suggestions?

 


I don't know if this could be made useful but since it's free...
Microsoft makes a free application called Synctoy that can be asked to
run a checksum on each file in a pair of folders and make sure they
match. The application does not display the checksums it just uses them
to sync the folders and can be asked to do various things when it finds
discrepancies. On the plus side, it's free. --J.B.

Jeff Hedberg jeff@unioneditorial.com [Avid-L2] wrote:
>
>
> You could choose the folder containing all the folders as a 'source'
> and then create a new folder on the other drive as the destination.
> However, in seeing your other comments - this is NOT going to give you
> a list of all the files with a checksum for each one (like you
> described in a different post). I agree that something like 'shotput'
> that Bogdan mentioned is what the studio is probably expecting you to use.
>
> Rsync is great for your own peace of mind, but there's no report of
> "this file has this checksum" - and it sounds like you need that feature.
>
> After finding out the hard way that the MacOS Finder will sometimes
> just skip files in folders as I'm moving DPX sequences around, I now
> rely on rsync any time I deal with them. (I think the Finder freaks
> out when there are more than 1000 files in a folder).
>
> Jeff
>
>
> ------------------
> Jeff Hedberg
>
> Director of Operations
> Union Editorial
> 575 Broadway,6th floor
> New York, NY 10012
>
>> On Dec 7, 2016, at 6:51 PM, bigfish@pacbell.net
>> <mailto:bigfish@pacbell.net> [Avid-L2] <Avid-L2@yahoogroups.com
>> <mailto:Avid-L2@yahoogroups.com>> wrote:
>>
>> So I played with the front end software you suggested and I seem to
>> have copied a file. Now I have a raid tower with a folder containing
>> all the DPX files for my show. The folder is approx 4 TB of dpx files
>> numbers sequentially. How would I use arSync to copy this folder to
>> another drive? Would I have to .zip the folder and then add that as
>> the source in arSync then choose a destination location? Would I
>> have to create a target folder on the destination drive or could I
>> just choose the drive and have the source file(s) end up on the
>> surface of the drive? The thought of having to select all the DPX
>> files in the folder seem like a pain hence why I think it was
>> suggested in this thread to .zip a folder unless I misunderstood.
>>
>>
>>
>> ---InAvid-L2@yahoogroups.com <mailto:Avid-L2@yahoogroups.com>,
>> <jeff@...> wrote :
>>
>> Hi John,
>> I've solved this in the past by using the command line to copy the
>> files using the command 'rsync' that has an option to checksum every
>> file after the copy confirming that it is the same on both sides.
>>
>> I definitely don't consider myself a rocket scientist, but if you
>> want to stay out of the command line, perhaps you could try a program
>> called 'arRsync' (which is a graphical front end for rsync). I just
>> downloaded it from
>> here:https://sourceforge.net/projects/arrsync/There is an 'options'
>> pane where you can change the default of 'Modification Date and Size
>> Check' to 'Always Checksum'. This should give you what you are after.
>> You'll notice the copies take longer than normal, but that is your
>> checksum doing its thing!
>>
>> Although, after re-reading... are you looking for a tool that will
>> give you a checksum for each individual DPX file? I have NOT had to
>> deliver like that yet, and will admit I'm not entirely sure how I'd
>> attempt that. I know when I download large individual files I can get
>> and MD5 checksum to confirm that I have the whole thing, but I can't
>> imagine doing that for each DPX file. Do they let you .zip a folder's
>> worth of files and give you checksum on that instead?
>>
>> Not sure if I've been helpful or not - but I'm sending it anyways!
>>
>> Jeff
>>
>> ------------------
>> Jeff Hedberg
>>
>> Director of Operations
>> Union Editorial
>> 575 Broadway,6th floor
>> New York, NY 10012
>>
>>> On Dec 7, 2016, at 2:13 PM, John Moorebigfish@...
>>> <mailto:bigfish@...>[Avid-L2] <Avid-L2@yahoogroups.com
>>> <mailto:Avid-L2@yahoogroups.com>> wrote:
>>>
>>>
>>> Part of a network delivery is all kinds of archival masters in
>>> DPX so the files are huge. They also request a check sum. I
>>> understand just the basics that a check sum will assure the data
>>> was copied correctly. I know the DITs have used Silverlight
>>> software to provide us with the field media and that software
>>> offers check sum type stuff and other things.
>>>
>>> Can anyone suggest a mac compatible Check Sum software that
>>> doesn't take a rocket scientist to set up? I have seen a friend
>>> go to the mac terminal and run a check sum there where he could
>>> confirm a .mov file was identical to the original even if the
>>> file name had been changed. I think the network wants something
>>> more sophisticated than that.
>>>
>>> Having never dealt with this I am just starting to research our
>>> options. Any experience in this area will be appreciated.
>>> John Moore Barking Trout Productions Studio City, CAbigfish@...
>>> <mailto:bigfish@...>
>>>
>>
>>
>
>
>
>

__._,_.___

Posted by: john beck <jb30343@windstream.net>
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (12)

Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.

this is the Avid-L2

.

__,_._,___

Re: [Avid-L2] Drop Frame DPX?

 

You can think of DPX as just an image sequence. Each frame increases its number sequentially, never skipping a number. Those filenames don't relate back to timecode. Just the order of every frame. No matter what framerate the project is, the DPX files will all be labeled sequentially (no break in numbers)


If you were to take these files in to something like After Effects, After Effects wouldn't have the ability to tell what the framerate of the sequence is. You'd have to tell it yourself. From there it could put timecode on it for you.

This is one of the reasons why when you talk w/ Nuke or Flame or 3D artists they will often talk about frame number rather than a timecode of the area they are working on.

This used to come up more when we were cutting SD in 29.97, but sourcing from 24fps film... Flame would need to remove the pulldown to work on shots... so now it's got a new framerate and new timecode... anyways, I often remember discussions similar to: "... on the fifth frame of that shot..." rather than: "...on 02:15:12:10..." 

Jeff

------------------
Jeff Hedberg

Director of Operations
Union Editorial
575 Broadway,6th floor
New York, NY 10012

On Dec 7, 2016, at 7:37 PM, John Moore bigfish@pacbell.net [Avid-L2] <Avid-L2@yahoogroups.com> wrote:


With all the fun things I'm learning about 4K and DPX exports from Avid I just had a thought about Drop Frame in a DPX world.  So far all my 4K is 23.976 so there is no Drop Frame issues.  I export basing the frame count on sequence time code which is very helpful when making patch fixes etc...  

I haven't tried this but does anyone know if I was working in a 4K project with 29.97 frame rate where drop frame was used.  If I export a DPX with frame count based on time code what happens at the drop frames?  Would the break in sequential numbers cause the DPX to burp?  I know it is possible to just have the frame count ascend and ignore time code but where's the fun in that.  Basing the frame count on time code is really helpful.  Anybody know if a DPX with frame count based on drop frame time code would work correctly?
 
John Moore Barking Trout Productions Studio City, CA bigfish@pacbell.net


__._,_.___

Posted by: Jeff Hedberg <jeff@unioneditorial.com>
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (2)

Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.

this is the Avid-L2

.

__,_._,___

Re: [Avid-L2] OTish: Software for running Checksum on Data Copies Suggestions?

 

You could choose the folder containing all the folders as a 'source' and then create a new folder on the other drive as the destination.

However, in seeing your other comments - this is NOT going to give you a list of all the files with a checksum for each one (like you described in a different post). I agree that something like 'shotput' that Bogdan mentioned is what the studio is probably expecting you to use.

Rsync is great for your own peace of mind, but there's no report of "this file has this checksum" - and it sounds like you need that feature.

After finding out the hard way that the MacOS Finder will sometimes just skip files in folders as I'm moving DPX sequences around, I now rely on rsync any time I deal with them. (I think the Finder freaks out when there are more than 1000 files in a folder).

Jeff


------------------
Jeff Hedberg

Director of Operations
Union Editorial
575 Broadway,6th floor
New York, NY 10012

On Dec 7, 2016, at 6:51 PM, bigfish@pacbell.net [Avid-L2] <Avid-L2@yahoogroups.com> wrote:

So I played with the front end software you suggested and I seem to have copied a file.  Now I have a raid tower with a folder containing all the DPX files for my show. The folder is approx 4 TB of dpx files numbers sequentially.  How would I use arSync to copy this folder to another drive?  Would I have to .zip the folder and then add that as the source in arSync then choose a destination location?  Would I have to create a target folder on the destination drive or could I just choose the drive and have the source file(s) end up on the surface of the drive?  The thought of having to select all the DPX files in the folder seem like a pain hence why I think it was suggested in this thread to .zip a folder unless I misunderstood.



---In Avid-L2@yahoogroups.com, <jeff@...> wrote :

Hi John,
I've solved this in the past by using the command line to copy the files using the command 'rsync' that has an option to checksum every file after the copy confirming that it is the same on both sides. 

I definitely don't consider myself a rocket scientist, but if you want to stay out of the command line, perhaps you could try a program called 'arRsync' (which is a graphical front end for rsync). I just downloaded it from here: https://sourceforge.net/projects/arrsync/ There is an 'options' pane where you can change the default of 'Modification Date and Size Check' to 'Always Checksum'. This should give you what you are after. You'll notice the copies take longer than normal, but that is your checksum doing its thing!

Although, after re-reading... are you looking for a tool that will give you a checksum for each individual DPX file? I have NOT had to deliver like that yet, and will admit I'm not entirely sure how I'd attempt that. I know when I download large individual files I can get and MD5 checksum to confirm that I have the whole thing, but I can't imagine doing that for each DPX file. Do they let you .zip a folder's worth of files and give you checksum on that instead?

Not sure if I've been helpful or not - but I'm sending it anyways!

Jeff

------------------
Jeff Hedberg

Director of Operations
Union Editorial
575 Broadway,6th floor
New York, NY 10012

On Dec 7, 2016, at 2:13 PM, John Moore bigfish@... [Avid-L2] <Avid-L2@yahoogroups.com> wrote:


Part of a network delivery is all kinds of archival masters in DPX so the files are huge.  They also request a check sum.  I understand just the basics that a check sum will assure the data was copied correctly.  I know the DITs have used Silverlight software to provide us with the field media and that software offers check sum type stuff and other things.

Can anyone suggest a mac compatible Check Sum software that doesn't take a rocket scientist to set up?  I have seen a friend go to the mac terminal and run a check sum there where he could confirm a .mov file was identical to the original even if the file name had been changed.  I think the network wants something more sophisticated than that. 

Having never dealt with this I am just starting to research our options.  Any experience in this area will be appreciated.
 
John Moore Barking Trout Productions Studio City, CA bigfish@...




__._,_.___

Posted by: Jeff Hedberg <jeff@unioneditorial.com>
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (11)

Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.

this is the Avid-L2

.

__,_._,___

Wednesday, December 7, 2016

[Avid-L2] Drop Frame DPX?

 

With all the fun things I'm learning about 4K and DPX exports from Avid I just had a thought about Drop Frame in a DPX world.  So far all my 4K is 23.976 so there is no Drop Frame issues.  I export basing the frame count on sequence time code which is very helpful when making patch fixes etc... 

I haven't tried this but does anyone know if I was working in a 4K project with 29.97 frame rate where drop frame was used.  If I export a DPX with frame count based on time code what happens at the drop frames?  Would the break in sequential numbers cause the DPX to burp?  I know it is possible to just have the frame count ascend and ignore time code but where's the fun in that.  Basing the frame count on time code is really helpful.  Anybody know if a DPX with frame count based on drop frame time code would work correctly?
 
John Moore Barking Trout Productions Studio City, CA bigfish@pacbell.net

__._,_.___

Posted by: John Moore <bigfish@pacbell.net>
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (1)

Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.

this is the Avid-L2

.

__,_._,___

Re: [Avid-L2] OTish: Software for running Checksum on Data Copies Suggestions?

 

So I played with the front end software you suggested and I seem to have copied a file.  Now I have a raid tower with a folder containing all the DPX files for my show. The folder is approx 4 TB of dpx files numbers sequentially.  How would I use arSync to copy this folder to another drive?  Would I have to .zip the folder and then add that as the source in arSync then choose a destination location?  Would I have to create a target folder on the destination drive or could I just choose the drive and have the source file(s) end up on the surface of the drive?  The thought of having to select all the DPX files in the folder seem like a pain hence why I think it was suggested in this thread to .zip a folder unless I misunderstood.



---In Avid-L2@yahoogroups.com, <jeff@...> wrote :

Hi John,
I've solved this in the past by using the command line to copy the files using the command 'rsync' that has an option to checksum every file after the copy confirming that it is the same on both sides. 

I definitely don't consider myself a rocket scientist, but if you want to stay out of the command line, perhaps you could try a program called 'arRsync' (which is a graphical front end for rsync). I just downloaded it from here: https://sourceforge.net/projects/arrsync/ There is an 'options' pane where you can change the default of 'Modification Date and Size Check' to 'Always Checksum'. This should give you what you are after. You'll notice the copies take longer than normal, but that is your checksum doing its thing!

Although, after re-reading... are you looking for a tool that will give you a checksum for each individual DPX file? I have NOT had to deliver like that yet, and will admit I'm not entirely sure how I'd attempt that. I know when I download large individual files I can get and MD5 checksum to confirm that I have the whole thing, but I can't imagine doing that for each DPX file. Do they let you .zip a folder's worth of files and give you checksum on that instead?

Not sure if I've been helpful or not - but I'm sending it anyways!

Jeff

------------------
Jeff Hedberg

Director of Operations
Union Editorial
575 Broadway,6th floor
New York, NY 10012

On Dec 7, 2016, at 2:13 PM, John Moore bigfish@... [Avid-L2] <Avid-L2@yahoogroups.com> wrote:


Part of a network delivery is all kinds of archival masters in DPX so the files are huge.  They also request a check sum.  I understand just the basics that a check sum will assure the data was copied correctly.  I know the DITs have used Silverlight software to provide us with the field media and that software offers check sum type stuff and other things.

Can anyone suggest a mac compatible Check Sum software that doesn't take a rocket scientist to set up?  I have seen a friend go to the mac terminal and run a check sum there where he could confirm a .mov file was identical to the original even if the file name had been changed.  I think the network wants something more sophisticated than that. 

Having never dealt with this I am just starting to research our options.  Any experience in this area will be appreciated.
 
John Moore Barking Trout Productions Studio City, CA bigfish@...


__._,_.___

Posted by: bigfish@pacbell.net
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (10)

Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.

this is the Avid-L2

.

__,_._,___

Re: [Avid-L2] When AMA haunts a timeline...

 

That should have been Users/Shared/AvidMediaComposer/AMA Management/AMA Metadata Folders
 
On Wed, Dec 7, 2016, at 06:22 PM, Dan McCabe danlist@bestmail.us [Avid-L2] wrote:
 

 

Thanks! Those are the folders in Users/Shared/AvidMediaComposer/AMA Management ?
Am I free to delete everything including Managed AMA Media Folder.xml?
 
On Wed, Dec 7, 2016, at 05:57 PM, Bogdan Grigorescu wrote:
rename or delete AMA database(s)
 
BG
www.finale.tv

 

From: "Dan McCabe danlist@bestmail.us [Avid-L2]" <Avid-L2@yahoogroups.com>
To: Avid List 2 <Avid-L2@yahoogroups.com>
Sent: Wednesday, December 7, 2016 2:45 PM
Subject: [Avid-L2] When AMA haunts a timeline...
 
 
In general, I always AMA footage into the Avid (MC 8.5.2) then transcode
or consolidate depending on circumstances. I always delete the original
AMA clips.

Here's where the AMA files come back from the dead:

If the drives that have the original footage remain available to the
Avid, every so often the sequence randomly reverts some of the clips
back to the original AMA files. It will be one clip out of ten in a bin
in which all of them were brought in at the same time.

If I try to relink the sequence, nothing happens. If I match back to the
bin and try to relink the specific clip to the specific drive where I
know the media resides, I can relink it. But it's a pain...

Does this happen to anyone else? Is there a setting somewhere I should
turn on and off that allows AMA?

Thanks for any advice.

 

 

__._,_.___

Posted by: Dan McCabe <danlist@bestmail.us>
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (4)

Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.

this is the Avid-L2

.

__,_._,___