staff
1,114
edits
Avdump3: Difference between revisions
→Reporting: PrintHashes requires Consumers to be set before it will print anything
(→Reporting: PrintHashes requires Consumers to be set before it will print anything) |
|||
| (43 intermediate revisions by 6 users not shown) | |||
| Line 2: | Line 2: | ||
== Quickstart == | == Quickstart == | ||
{{eyecatch|Important|[[Avdump_GUI|Avdump GUI]] is a GUI version of Avdump3, for command line skeptics.}} | |||
Since you’re here you probably just want to send AVMF Packages (or Dumps) to AniDB which will add metadata to previously (or soon to be) added files. If that doesn’t mean anything to you, you probably want to start with | Since you’re here you probably just want to send AVMF Packages (or Dumps) to AniDB which will add metadata to previously (or soon to be) added files. If that doesn’t mean anything to you, you probably want to start with [[Content:Files]] and [[Auto-creqing]] and [[Tutorial:How to Add Files for Dummies]]. | ||
Anyway lets keep this short:<br /> | Anyway lets keep this short:<br /> | ||
# Login into AniDB with your account and visit https://anidb.net/user/setting, there | # Login into AniDB with your account and visit https://anidb.net/user/setting, there go to the Account tab and set a password for “UDP API Key”<br /> | ||
# Make sure [https://dotnet. | # Make sure [https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=6.0.0&arch=x64&rid=win10-x64 .Net 6.0] or higher is installed<br /> | ||
# Download the latest Version of AVD3 with the link in the section below and extract it somewhere.<br /> | # Download the latest Version of AVD3 with the link in the section below and extract it somewhere.<br /> | ||
# Start a terminal and navigate to AVDump3<br /> | # Start a terminal and navigate to AVDump3<br /> | ||
| Line 14: | Line 14: | ||
#: '''Linux:''' <code>dotnet AVDump3CL.dll --Auth=<YourUserName>:<YourUdpApiPassword> <APathToTheFiles></code> | #: '''Linux:''' <code>dotnet AVDump3CL.dll --Auth=<YourUserName>:<YourUdpApiPassword> <APathToTheFiles></code> | ||
# Optional: See sections below to improve usage experience | # Optional: See sections below to improve usage experience | ||
## For example, to avoid dumping the same files over and over again, add the following parameter: <code>--DoneLogPath=done.txt</code> | |||
# Optional: Strongly consider adding --UploadErrors to the arguments. For more information see https://wiki.anidb.net/Avdump3#UploadErrors_Argument | |||
== Download == | == Download == | ||
[ | [https://cdn.anidb.net/client/avdump3/avdump3_8293_stable.zip DOWNLOAD (ZIP)] (Build 8293) MD5 = 87ed86a9babc172d638071ff1123fdc7 | ||
== What’s new compared to AVD2? == | == What’s new compared to AVD2? == | ||
AVD3 is a complete rewrite of AVD2 which uses .NET | AVD3 is a complete rewrite of AVD2 which uses .NET 6.0 instead of .NET Framework and treats Linux as a first class citizen. So it should run on Linux just as well as on Windows. To name a few other big differences: | ||
* Can process multiple files in parallel | * Can process multiple files in parallel | ||
| Line 27: | Line 29: | ||
* More efficient reading | * More efficient reading | ||
* Can move/rename files based on scripts | * Can move/rename files based on scripts | ||
* Redone | * Redone command line arguments | ||
* More hash algorithms | * More hash algorithms | ||
* Latest version of MediaInfoLib (MIL) is being used (currently | * Latest version of MediaInfoLib (MIL) is being used (currently 2109) | ||
* Support for .vtt subtitle files | * Support for .vtt subtitle files | ||
* Support for 32bit has been dropped | * Support for 32bit has been dropped | ||
* Support for Linux aarch64 has been added | |||
* Support for MacOS is not yet available (To add support https://github.com/DvdKhl/AVDump3/blob/master/AVDump3NativeLib/src/AVD3MirrorBuffer.c needs to be implemented, help would be appreciated) | * Support for MacOS is not yet available (To add support https://github.com/DvdKhl/AVDump3/blob/master/AVDump3NativeLib/src/AVD3MirrorBuffer.c needs to be implemented, help would be appreciated) | ||
== Supported formats == | == Supported formats == | ||
Most major file formats are supported to some degree. While hashes are created for all file types, stream details remain | Most major file formats are supported to some degree. While hashes are created for all file types, stream details remain shaky for others (notably 'swf'). Because of that for some filetypes (underlined) only the hashes get auto-creqed. | ||
* '''Video files:''' asf/wmv, avi, flv, m2ts, mk3d, mkv, mov, mp4, mpg/mpeg, ogm, ogv, qt, rm/rmvb, <u>swf</u>, ts, webm | * '''Video files:''' asf/wmv, avi, flv, m2ts, mk3d, mkv, mov, mp4, mpg/mpeg, ogm, ogv, qt, rm/rmvb, <u>swf</u>, ts, webm | ||
| Line 44: | Line 47: | ||
== UploadErrors Argument == | == UploadErrors Argument == | ||
When enabled errors while processing or program crashes are uploaded to AniDB which makes it far easier to discover and fix bugs. | |||
Error reports sent look like the following: | |||
<AVD3CLException thrownOn="2020-08-11 18:31:34.2837"> | |||
<Information> | |||
<EntryAssemblyVersion>3.0.8163.0</EntryAssemblyVersion> | |||
<LibVersion>3.0.8163.0</LibVersion> | |||
<Session>395e5893-145d-4a19-8fc9-1ddc0f7feed6</Session> | |||
<Framework>3.1.7</Framework> | |||
<OSVersion>Microsoft Windows NT 6.2.9200.0</OSVersion> | |||
<IntPtr.Size>8</IntPtr.Size> | |||
<Is64BitOperatingSystem>true</Is64BitOperatingSystem> | |||
<Is64BitProcess>true</Is64BitProcess> | |||
<ProcessorCount>40</ProcessorCount> | |||
<UserInteractive>true</UserInteractive> | |||
<SystemPageSize>4096</SystemPageSize> | |||
<WorkingSet>239337472</WorkingSet> | |||
</Information> | |||
<Message>CreatingInfoProviders</Message> | |||
<Data> | |||
<FileName>Hidden(7E940D28B4A58A87C4C19B39EB6EA5E2B11CC7B13762DE44BB7338E752F6676722B227DBA942302F956AF1CE040C57D9295CF53AB33B5DE45FD4FAB6FB462E47)</FileName> | |||
</Data> | |||
<Cause> | |||
<InvalidOperationException> | |||
<Message>MediaInfoLib couldn't open the file</Message> | |||
<Stacktrace> | |||
<Frame>at AVDump3Lib.Information.InfoProvider.MediaInfoLibProvider..ctor(String filePath) in D:\Projects\C#\AVDump3\AVDump3Lib\Information\InfoProvider\MediaInfoLibProvider.cs:line 522</Frame> | |||
<Frame>at AVDump3Lib.Information.AVD3InformationModule.<>c.<.ctor>b__4_4(InfoProviderSetup setup) in D:\Projects\C#\AVDump3\AVDump3Lib\Information\AVD3InformationModule.cs:line 30</Frame> | |||
<Frame>at AVDump3Lib.Information.InfoProvider.InfoProviderFactory.Create(InfoProviderSetup setup) in D:\Projects\C#\AVDump3\AVDump3Lib\Information\InfoProvider\InfoProviderFactory.cs:line 40</Frame> | |||
<Frame>at AVDump3CL.AVD3CLModule.<>c__DisplayClass34_0.<CreateFileMetaInfo>b__0(IInfoProviderFactory x) in D:\Projects\C#\AVDump3\AVDump3CL\AVD3CLModule.cs:line 449</Frame> | |||
<Frame>at System.Linq.Enumerable.SelectIListIterator`2.ToArray()</Frame> | |||
<Frame>at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)</Frame> | |||
<Frame>at AVDump3CL.AVD3CLModule.CreateFileMetaInfo(String filePath, ImmutableArray`1 blockConsumers) in D:\Projects\C#\AVDump3\AVDump3CL\AVD3CLModule.cs:line 449</Frame> | |||
</Stacktrace> | |||
<Data /> | |||
</InvalidOperationException> | |||
</Cause> | |||
<Stacktrace /> | |||
</AVD3CLException> | |||
The value within Hidden() is a hash so it is not possible to get the original data back from that value. When --IncludePersonalData is added those values will be shown in clear text. | |||
'''Attention:''' Please be aware that when --IncludePersonalData is active as well, additional data like the commandline arguments (except for passwords) and path information is sent as well! | |||
== Arguments == | == Arguments == | ||
| Line 219: | Line 265: | ||
| | | | ||
| Print calculated hashes in hexadecimal format to console | | Print calculated hashes in hexadecimal format to console | ||
Consumers must be specified for the the PrintHashes command to output anything | |||
| style="white-space: nowrap;" | --PrintHashes | | style="white-space: nowrap;" | --PrintHashes | ||
| style="white-space: nowrap;" | False | | style="white-space: nowrap;" | False | ||
| Line 238: | Line 285: | ||
| Reports will be saved to the specified directory | | Reports will be saved to the specified directory | ||
| style="white-space: nowrap;" | --ReportDirectory=<Directory> | | style="white-space: nowrap;" | --ReportDirectory=<Directory> | ||
| style="white-space: nowrap;" | <The directory AVD3 | | style="white-space: nowrap;" | <The directory AVD3 is invoked in> | ||
|- | |- | ||
| style="white-space: nowrap;" | --ReportFileName | | style="white-space: nowrap;" | --ReportFileName | ||
| Line 245: | Line 292: | ||
| style="white-space: nowrap;" | --ReportFileName=<FileName> | | style="white-space: nowrap;" | --ReportFileName=<FileName> | ||
| style="white-space: nowrap;" | ${FileName}.${ReportName}.${ReportFileExtension} | | style="white-space: nowrap;" | ${FileName}.${ReportName}.${ReportFileExtension} | ||
|- | |||
| style="white-space: nowrap;" | --ReportContentPrefix | |||
| | |||
| Each report will be prefixed with the arguments content.<br />See ReportFileName for Placeholders. | |||
| style="white-space: nowrap;" | --ReportContentPrefix=${FullName} | |||
| style="white-space: nowrap;" | | |||
|- | |- | ||
| style="white-space: nowrap;" | --ExtensionDifferencePath | | style="white-space: nowrap;" | --ExtensionDifferencePath | ||
| Line 291: | Line 344: | ||
| Various places may include personal data. Currently this only affects error files, which will then include the full filepath | | Various places may include personal data. Currently this only affects error files, which will then include the full filepath | ||
| style="white-space: nowrap;" | --IncludePersonalData | | style="white-space: nowrap;" | --IncludePersonalData | ||
| style="white-space: nowrap;" | | | style="white-space: nowrap;" | False | ||
|- | |||
| style="white-space: nowrap;" | --PrintDiscoveredFiles | |||
| | |||
| Print each discovered file path instead of printing the count | |||
| style="white-space: nowrap;" | --PrintDiscoveredFiles | |||
| style="white-space: nowrap;" | <The directory AVD3 is invoked in> | |||
|- | |- | ||
| style="white-space: nowrap;" | --ErrorDirectory | | style="white-space: nowrap;" | --ErrorDirectory | ||
| Line 297: | Line 356: | ||
| If --SaveErrors is specified the error files will be placed in the specified path | | If --SaveErrors is specified the error files will be placed in the specified path | ||
| style="white-space: nowrap;" | --ErrorDirectory=<DirectoryPath> | | style="white-space: nowrap;" | --ErrorDirectory=<DirectoryPath> | ||
| style="white-space: nowrap;" | <The directory AVD3 | | style="white-space: nowrap;" | <The directory AVD3 is invoked in> | ||
|- | |- | ||
| style="white-space: nowrap;" | --NullStreamTest | | style="white-space: nowrap;" | --NullStreamTest | ||
| Line 452: | Line 511: | ||
==Changelog== | ==Changelog== | ||
<div style="height: 280px; overflow: auto; padding: 3px; border:1px solid #AAAAAA; font-size: small"> | <div style="height: 280px; overflow: auto; padding: 3px; border:1px solid #AAAAAA; font-size: small"> | ||
'''0.1.8293.0 : 2021.12.02''' | |||
* Updated dependency from .Net 5.0 to .Net 6.0 | |||
* Added support for Linux aarch64 | |||
'''0.1.8238.0 : 2021.10.31''' | |||
* Updated dependency from .Net Core 3.1 to .Net 5.0 | |||
* Updated to MediaInfoLib 2109 | |||
* Fixed various small bugs | |||
'''0.1.8213.0 : 2020.09.27''' | |||
* Fixed ED2k hash calculation for files with a file size that is a multiple of 9728000 bytes (Issue #48) | |||
** Included alternative ED2k link in ed2k log | |||
** Added <ed2k_alt/> node to output xml | |||
* Don't crash when a file to be processed cannot be opened (Issue #53) | |||
* Switched from FileShare.Read to FileShare.ReadWrite for writing to files | |||
* Added TiB/s | |||
* Set UTF8 as Console output when UTF8OUT is added as an arg (console wide!) | |||
* Added PARSEARGS which will make AVD3 parse the provided arguments itself (windows only), fixing the double quote escape problem | |||
* Don't crash when the console cursor cannot be manipulated | |||
* Output final progress before termination | |||
* Fixed various small bugs | |||
'''0.1.8188.0 : 2020.08.23''' | |||
* Fixed various small bugs | |||
'''0.1.8187.0 : 2020.08.22''' | |||
* Fixed various small bugs | |||
'''0.1.8185.0 : 2020.08.17''' | |||
* Updated MIL to version 20.08 | |||
* Fixed various small bugs | |||
'''0.1.8173.0 : 2020.08.15''' | |||
* Fixed ReportFileName throwing an exception when used | |||
* EffectiveCommandLineArguments now with sub elements | |||
'''0.1.8167.0 : 2020.08.14''' | |||
'''0.1.8167.0 : 2020.08. | * Fixed crash while displaying the progress | ||
* Switched to portable pdb (linenumbers in linux) | |||
* Implemented "UserValueType" for setting properties (Issue #45) | * Implemented "UserValueType" for setting properties (Issue #45) | ||
** Passwords are now marked with PasswordType for UserValueType | ** Passwords are now marked with PasswordType for UserValueType | ||
| Line 479: | Line 574: | ||
* semi-public pre-release | * semi-public pre-release | ||
</div> | </div> | ||
[[Category:Avdump]] | |||