Analyzing Dynamo Scripts and Store Information in CSV
Dynamo Scripts are an integral part of automation within Autodesk Revit, Civil 3D, and other BIM software. These scripts facilitate tasks such as generating complex geometries, automating repetitive processes, and enhancing workflow efficiency. Structured as JSON files, Dynamo scripts provide a standardized format that is both machine-readable and easy to parse programmatically.
In this blog, we delve into two robust approaches to extract, analyze, and store information from Dynamo scripts into CSV files—using C# and Python. These methods cater to developers and BIM professionals who aim to streamline data analysis, transformation, and reporting. By the end of this article, you will have practical insights into leveraging both languages to process Dynamo script JSON files effectively and produce structured CSV outputs for further use.
Necessity of Extracting and Organizing Dynamo Script Data into CSV Format
Dynamo scripts are structured as JSON files, making them inherently easy to parse programmatically. However, the flexibility of JSON comes with challenges when organizing and documenting multiple scripts for larger projects. Extracting key information from Dynamo script files and storing it in a CSV format can greatly enhance workflow management, reporting, and project organization.
Benefits of Extracting Dynamo Script Information
- Centralized Metadata: Consolidating script details, such as names, descriptions, and dependencies, into a CSV enables quick access and analysis.
- Improved Organization: For projects involving numerous scripts, a CSV file serves as a comprehensive index, simplifying navigation and documentation.
- Compatibility with Tools: CSV files can be easily imported into spreadsheet software or data visualization tools, facilitating collaboration with non-technical team members.
- Enhanced Reporting: With key script metadata stored in a structured format, generating reports or summaries becomes straightforward.
Key Features of the Task
To extract and store Dynamo script information effectively, the process involves:
- Directory Traversal: Automatically navigating through folders and subfolders containing Dynamo JSON files.
- Metadata Extraction: Capturing essential details, including script names, descriptions, node dependencies, and modification timestamps.
- Structured Storage: Writing the extracted data into a CSV file, ensuring ease of use for downstream workflows.
Method 1: Using C#
C# offers robust tools for handling files and data. Below is a complete solution to traverse a directory, process JSON files, and save their data in a CSV file.
Steps in C#:
- Traverse all files in the given directory (including subfolders).
- Parse the JSON content to extract specific fields.
- Retrieve metadata such as file modification dates.
- Write the extracted information into a CSV file.
C# Code
1using System;
2using Newtonsoft.Json;
3using Newtonsoft.Json.Linq;
4
5namespace dynamoscriptanalyzer;
6
7public class Analyzer
8{
9 public static void AnalyzeFolder(string inputFolderPath, string outputFilePath){
10 try {
11 // Prepare CSV handler
12 var csvHeader = "FolderPath,Name,Description,DynamoVersion,DynamoPlayer,PythonScript,ExternalPackages,LstModified";
13 var csvData = new List<string> {csvHeader};
14
15 // Get all DYN files in the folder
16 var dynFiles = Directory.GetFiles(inputFolderPath, "*.dyn", SearchOption.AllDirectories);
17
18 foreach (var file in dynFiles) {
19 try {
20 //Get folder name
21 string folderName = Path.GetDirectoryName(file) ?? "Root";
22
23 // Read and parse DYN
24 string jsonContent = File.ReadAllText(file);
25 JObject script = JObject.Parse(jsonContent);
26
27 // Extract required data
28 string name = script.SelectToken("Name")?.ToString() ?? "N/A";
29 string description = script.SelectToken("Description")?.ToString() ?? "N/A";
30
31 // Get Dynamo version
32 string dynamoVersion = script.SelectToken("View.Dynamo.Version")?.ToString() ?? "Unknown";
33
34 // Check Inputs
35 var inputs = script.SelectToken("Inputs")?.ToObject<List<JToken>>() ?? new List<JToken>();
36 bool isDynamoPlayerReady = inputs.Count > 0;
37
38 // Check Nodes for PythonScriptNode
39 var nodes = script.SelectToken("Nodes")?.ToObject<List<JObject>>() ?? new List<JObject>();
40 bool hasPythonScriptNode = nodes.Exists(node => node.SelectToken("NodeType")?.ToString() == "PythonScriptNode");
41
42 // Check NodeLibraryDependencies for Packages
43 var dependencies = script.SelectToken("NodeLibraryDependencies")?.ToObject<List<JObject>>() ?? new List<JObject>();
44 var packageNames = dependencies
45 .Where(dep => dep.SelectToken("ReferenceType")?.ToString() == "Package")
46 .Select(dep => dep.SelectToken("Name")?.ToString())
47 .ToList();
48
49 // Get file's last modified date
50 string lastModified = File.GetLastWriteTime(file).ToString("yyyy-MM-dd");
51
52 // Add rows to CSV
53 var row = $"{folderName},{name},{description},{dynamoVersion},{isDynamoPlayerReady},{hasPythonScriptNode},\"{string.Join(";", packageNames)}\",{lastModified}";
54 csvData.Add(row);
55
56 System.Console.WriteLine($"Processed file: {Path.GetFileName(file)}");
57 }
58 catch (Exception ex){
59 System.Console.WriteLine($"Error processing file {file}: {ex.Message}");
60
61 }
62 }
63 // Save all data to CSV
64 File.WriteAllLines(outputFilePath, csvData);
65 System.Console.WriteLine($"CSV saved successfully at {outputFilePath}");
66 }
67 catch (Exception ex) {
68 System.Console.WriteLine($"Error analysing folder: {ex.Message}");
69 }
70 }
71
72}Explanation of Key Features
- Folder Traversal:
- Using
Directory.GetFileswithSearchOption.AllDirectoriesto include subfolders.
- Using
- JSON Parsing:
- Utilized
Json.NETfor easy parsing and querying of JSON data.
- Utilized
- CSV Writing:
- Leveraged
File.WriteAllLinesfor efficient CSV writing.
- Leveraged
Method 2: Using Python
Python provides a simpler syntax and built-in libraries for file and data handling. Here’s how to achieve the same task in Python.
Steps in Python:
- Use
os.walkto iterate through directories. - Parse JSON data using the
jsonmodule. - Extract and process specific fields.
- Write the processed data to a CSV file.
Python Code
1import os
2import json
3import csv
4from datetime import datetime
5
6def analyze_folder_and_save_to_csv(input_folder_path, output_file_path):
7 # Prepare CSV header
8 csv_header = [
9 "FolderPath",
10 "Name",
11 "Description",
12 "DynamoPlayer",
13 "PythonScript",
14 "ExternalPackages",
15 "LastModified",
16 "DynamoVersion"
17 ]
18
19 csv_data = []
20
21 # Walk through all the files and subfolders
22 for root, _, files in os.walk(input_folder_path):
23 for file in files:
24 if file.endswith(".dyn"):
25 try:
26 file_path = os.path.join(root, file)
27
28 # Read and parse JSON
29 with open(file_path, "r", encoding="utf-8") as json_file:
30 script = json.load(json_file)
31
32 # Extract required data
33 folder_path = root
34 name = script.get("Name", "N/a")
35 description = script.get("Description", "N/A")
36
37 # Check Inputs
38 inputs = script.get("Inputs", [])
39 is_dynamo_player_ready = len(inputs) > 0
40
41 # Check Nodes for PythonScriptNode
42 nodes = script.get("Nodes", [])
43 has_python_script_node = any(
44 node.get("NodeType") == "PythonScriptNode" for node in nodes
45 )
46
47 # Check NodeLibraryDependencies for Packages
48 dependencies = script.get("NodeLibraryDependencies", [])
49 package_names = [
50 dep.get("Name", "") for dep in dependencies
51 if dep.get("ReferenceType") == "Package"
52 ]
53
54 # Get Dynamo Version
55 dynamo_version = script.get("View", {}).get("Dynamo", {}).get("Version", "Unknown")
56
57 # Get file's last modified date
58 last_modified = datetime.fromtimestamp(
59 os.path.getmtime(file_path)
60 ).strftime("%Y-%m-%d %H:%M:%S")
61
62 # Add row to CSV
63 csv_data.append([
64 folder_path,
65 name,
66 description,
67 is_dynamo_player_ready,
68 has_python_script_node,
69 ";".join(package_names),
70 last_modified,
71 dynamo_version
72 ])
73
74 print(f"Processed file: {file}")
75
76 except Exception as ex:
77 print(f"Error processing file {file}: {ex}")
78
79 # Write data to CSV
80 with open(output_file_path, "w", encoding="utf-8", newline="") as csv_file:
81 writer = csv.writer(csv_file)
82 writer.writerow(csv_header)
83 writer.writerows(csv_data)
84
85 print (f"CSV saved successfully at {output_file_path}")
86
87# Main execution
88if __name__ == "__main__":
89 input_folder_path = "scripts"
90 output_file_path = "dynamo_script_analysis_py.csv"
91
92 analyze_folder_and_save_to_csv(input_folder_path=input_folder_path, output_file_path=output_file_path)
93Key Features
- Ease of Traversal:
- Python’s
os.walksimplifies folder traversal.
- Python’s
- JSON Handling:
- The built-in
jsonmodule makes it easy to load and process JSON files.
- The built-in
- CSV Writing:
- Python’s
csvmodule provides intuitive methods to write data.
- Python’s
Conclusion
Both C# and Python provide powerful tools to generate CSV files from JSON data. While C# is great for robust, enterprise-grade applications, Python’s simplicity makes it a good choice for quick data processing tasks. Choose the language that best fits your workflow, and start transforming your data today!