Thursday, 21 November 2013

Perl Automation

Listed below are few features which we can do with Perl.

It could be an automated spreadsheet related work (for Managers, Finance, Admin), XML/HTML content conversion (Developers) or Monitoring tools (for Managers level) etc.

Basically Perl will work well with Content management system (CMS), Sever side scripting, Test automation, Reporting activities etc.

Content related activities on XML & HTML:

Content Extraction
  • Search and replace – Pattern Matching with regular expression
  • XML Parser
  • Tag validation – test automation
  • XML to HTML conversion vice versa
  • Multiple XMLs to single  XML (Merging)
  • Single XML to multiple XML (Splitting)
  • Xml to NLM complied xml and displayed it as HTML pages
  • Tags Extraction from XML and HTML
  • XML / HTML file Validation
  • XML structure Validation - format flow - folder creation as per order

Spreadsheet Automation:
  • Take out details from single spreadsheet and update multiple spreadsheets (condition based)
  • Extract details from multiple spreadsheet and create a new one.
  • Concatenation of two cells
  • Updating multiple spreadsheet with the help of master spreadsheet
  • Content validation within spreadsheets.

Monitoring and Analysis:
  • Server log error alert and send mail notification to the respective teams depends on the error message.
  • Real time file availability checking with the help of CGI perl (web page)
  • Error capturing tool and displaying on web page (CGI Perl)
  • Analyze error messages and trigger required scripts with automatic mail notification.
  • Housekeeping scripts (cleanup old logs based on timestamp and archive them the required logs to the defined location)
  • Periodic server load monitoring and automatic mail notification to the respective team if any high load
  • Alert if any unknown entry on the server log
  • Server Disk space alert
  • Monitoring MySQL Tables
  • Monitor folder for new files
  • Monitor Website performance and uptime (CGI)

WinZip Automation:
  • Move content from different source and zip the same (based on rule)
  • Validate zip (for corrupted file)
  • Extract zip to the specified location and copy the files to the different location (Based on rule)

Server Side Automation:
  • Batch processing
  • File related activities (Update, Merge, Split..etc)
  • Alerting tools
  • Ref. Monitoring and Analysis

Content Extraction from Webpage:
  • Extract content from pages (Real time and libraries) even with authentication
  • Extract content from multiple web page and create files in the name of the title of the page
  • URL extraction
  • URL validation
  • Download files from multiple URLs

  • PDF split & Merge
  • PDF to Text files
  • PDF to XML files.
  • Validate PDF and report.

  • Extract width / height  from xml and validate physically present or not.
  • Extract image name from xml/html etc.. and pull the respective  from image bank.
  • Image Validation - Size capturing (width & height) and create the text files.
  • Alert if any unexpected image size

Files and Folder validation:
  • Search file from huge filesystem and Process
  • String comparisons
  • Test file attributes and then process

Database: (MySQL)
  • Monitoring tables
  • Query validation
  • CURD activities

  • Transform XML documents by using XSLT style sheets using Perl.
  • XML to any form of xml (well-formed/required xml format)

  • Download files from FTP and validate, Zip them and then Upload to other location (With mail notification)
  • Download Zip from FTP and then extract, move the files based on defined rule and then process. (With mail notification)
  • FTP Folders validation
  • To get rid of empty folders
  • To check the maximum size of all folder and alert if it exceed

  • Getting PUBMED ID from PubMed site (Elsevier is doing such process)


  1. Its great!! most of the automations are possible with Perl. and its nice to see the consolidated work and your expertise..