Command line tool for eZ Publish, called "eep"
By: Doug Plant | July 18, 2012 | eZ Publish development tips
Mugo has a tool that we use internally to help with the main aspects of a developer's life: development, debugging and maintenance. The tool is called "eep", short for "Ease eZ Publish". It's a command line tool which provides many functions and allows for accomplishing tasks quickly by integrating with tools like awk, grep and xargs for powerful one-liners and supports rapid development of powerful bash scripts by leveraging eep as a library of eZ Publish specific operations.
The details
eep is a standalone script, however it can hook in any 4.x kernel in order to operate on a particular eZ instance.
eep is intended to be useful to developers; not content managers or what have you. It is a little dangerous and focuses on getting the job done.
> cd http/dbx > eep use ezroot . msg:: Reseting ezroot > > ## Danger! Danger! > eep contentclass deleteclass folder
The output format is modeled after mysql output:
> eep contentnode info 8405 +--------------------------+----------------------------------+ I contentnode info [8405] | +--------------------------+----------------------------------+ I key | value | +--------------------------+----------------------------------+ | Name | Frequently Asked Questions | | ContentObjectID | 8503 | | MainNodeID | 8405 | | ClassIdentifier | downloadable_file | | PathIdentificationString | frequently_asked_questions | | PathString | /1/2/8405/ | | ParentNodeID | 2 | | CurrentLanguage | eng-US | | ContentObjectVersion | 1 | | RemoteID | 68e90a5f7fe2370a73374da2a3435634 | | IsHidden | 0 | | IsInvisible | 0 | | ContentObjectIsPublished | 1 | | Reverse related count | 1 | | Children count | 0 | | URL Alias | Frequently-Asked-Questions | +--------------------------+----------------------------------+
The basic concept of eep is to:
- provide a bunch of operations that you can do on an ezpublish instance
- focus on the ability to link operations together and use them with existing
- tools like grep, awk, xargs, sort, uniq, paste, wc, etc. to build up powerful one-liners
- support rapid development of shell scripts that would otherwise take hours to write and debug
The guiding values have thus far been:
- this is built for developers; it's a tool box for expert users
- don't duplicate functionalities available from other command line tools
- expect that the codebase exists as a catalog of PHP around the eZ Publish API
- make operations that focus on object and node IDs, which means that you can scale and multi-thread operations
eep is available on github at: https://github.com/mugoweb/eep
Installation is documented in install.txt and amounts to causing "eep" to simply invoke the script. Note that for linux installs, the command line completion stuff is very nice; even now with just basic functionality it's big help.
In addition to leveraging all the standard tools, making eep a command line tool provides a couple sweet benefits. The first is that spreading a heavy processing load across multiple CPUs is trivial. The second takes advantage of a custom daemon to schedule and control large lists of tasks; the daemon also provides operability via the eZ admin interface. We aren't releasing the daemon at this time; we are still rolling it into a production setting.
And now for some examples.
Example 1: Delete all unused content classes.
> eep list contentclasses +----------------------------+----+------+----------------------------------+--------+------------------------------------+---------+ I list content classes | +----------------------------+----+------+----------------------------------+--------+------------------------------------+---------+ I Identifier | Id | # | RemoteID | Lang | Name | Group | +----------------------------+----+------+----------------------------------+--------+------------------------------------+---------+ | announcement | 59 | 2 | 4508764cae43f93a9a17480685168739 | eng-US | Announcement | DBX | | article | 16 | 0 | c15b600eb9198b1924063b5a68758232 | eng-US | Article | | | asset_class | 57 | 5 | 851dade1c910ad26700b30a2c70ca4b2 | eng-US | Asset Class | Content | | banner | 45 | 10 | 9cb558e25fd946246bbb32950c00228e | eng-US | Banner | Content | | common_ini_settings | 14 | 1 | ffedf2e73b1ea0c3e630e42e2db9c900 | eng-US | Common ini settings | Setup | | custom_error | 91 | 2 | fe2fd148254033388798af7b6218d35d | eng-US | Custom Error | DBX | | deployment | 97 | 11 | 6e686a81fb81d07e9ff535effa86391f | eng-US | Deployment | Content | | disclaimer | 90 | 12 | 390e0f705816c632e56abd419c792e64 | eng-US | Disclaimer | Content | | product_disclosures | 73 | 11 | 4a1ed1df1cdb1846d6893a9b10c0687d | eng-US | Disclosures | Content | | product_distributions | 68 | 21 | 358c65fae841d33e28f4ee9476214058 | eng-US | Distributions | Content | | downloadable_file | 89 | 1393 | ceb50523639d1b2c1f5c76c817508ae6 | eng-US | Downloadable File | Media | | product_downloads | 70 | 53 | b4f38188ebe5d44d624c12890f83a20c | eng-US | Downloads | Content | | faq_item | 82 | 25 | b34c64ee3ee76964b25d8202eabd7e7f | eng-US | FAQ Item | DBX | | file | 28 | 1 | 637d58bfddf164627bdfd265733280a0 | eng-US | File | Media | | flash | 29 | 0 | 6cd17b98a41ee9355371a376e8868ee0 | eng-US | Flash | Media | | flash_recorder | 30 | 0 | e349c947fd306299418be35b07b9a940 | eng-US | Flash recorder | Media | | focus | 56 | 8 | be8710cb34466d9460e207c35d8e951b | eng-US | Focus | Content | | folder | 1 | 368 | a3d405b81be900468eb153d774f4f0d2 | eng-US | Folder | Content | | windows_media | 36 | 0 | 223dd2551e85b63b55a72d02363faab6 | eng-US | Windows media | Media | +----------------------------+----+------+----------------------------------+--------+------------------------------------+---------+
(note that the output is truncated for the blog post)
> eep list contentclasses | awk '$6==0' | article | 16 | 0 | c15b600eb9198b1924063b5a68758232 | eng-US | Article | | | flash | 29 | 0 | 6cd17b98a41ee9355371a376e8868ee0 | eng-US | Flash | Media | | flash_recorder | 30 | 0 | e349c947fd306299418be35b07b9a940 | eng-US | Flash recorder | Media | | gallery | 38 | 0 | 6a320cdc3e274841b82fcd63a86f80d1 | eng-US | Gallery | Content | | global_layout | 32 | 0 | f0271811b913befa8f062527e909f15e | eng-US | Global layout | Content | | infobox | 25 | 0 | 0b4e8accad5bec5ba2d430acb25c1ff6 | eng-US | Infobox | Content | | link | 34 | 0 | 74ec6507063150bc813549b22534ad48 | eng-US | Link | Content | | multicalendar | 26 | 0 | 99aec4e5682414517ed929ecd969439f | eng-US | Multicalendar | Content | | poll | 27 | 0 | 232937a3a2eacbbf24e2601aebe16522 | eng-US | Poll | Content | | quicktime | 35 | 0 | 16d7b371979d6ba37894cc8dc306f38f | eng-US | Quicktime | Media | | real_video | 37 | 0 | dba67bc20a4301aa04cc74e411310dfc | eng-US | Real video | Media | | silverlight | 47 | 0 | 8ab17aae77dd4f24b5a8e835784e96e7 | eng-US | Silverlight | Media | | flash_player | 31 | 0 | 20b2ed0982343e6e0a550f7f0c137e06 | eng-US | Video/Flash Player | Media | | windows_media | 36 | 0 | 223dd2551e85b63b55a72d02363faab6 | eng-US | Windows media | Media |
> eep list contentclasses | awk '$6==0 {print $2}'
article
flash
flash_recorder
gallery
global_layout
infobox
link
multicalendar
poll
quicktime
real_video
silverlight
flash_player
windows_media
> eep list contentclasses | awk '$6==0 {print $2}' | xargs -IOID eep contentclass deleteclass OID
Deleting 0 objects.
Done deleting objects.
PHP Fatal error: Call to a member function isClassAttributeRemovable() on a non-object in /home/dfp/http/dbx/kernel/classes/ezcontentclass.php on line 815
PHP Stack trace:
PHP 1. {main}() /home/dfp/eep/eep.php:0
PHP 2. require_once() /home/dfp/eep/eep.php:130
PHP 3. contentclass_commands->run() /home/dfp/eep/modules/contentclass/index.php:311
PHP 4. contentclass_commands->deleteClass() /home/dfp/eep/modules/contentclass/index.php:256
PHP 5. eZContentClass->remove() /home/dfp/eep/modules/contentclass/index.php:145
PHP 6. eZContentClass->isRemovable() /home/dfp/http/dbx/kernel/classes/ezcontentclass.php:758
PHP 7. eZContentClass->removableInformation() /home/dfp/http/dbx/kernel/classes/ezcontentclass.php:774
Fatal error: eZ Publish did not finish its request
The execution of eZ Publish was abruptly ended, the debug output is present below.
xargs: eep: exited with status 255; aborting
Well, that's annoying. I wanted to remove all the unused content classes; but it appears that the first useless content class has a problem. My guess is that it's because the "article" class makes use of an attribute that is not actually in the system.
> eep list attributes article +-----------------+--------+-----+------------+------+------+------+-----+-----------------+ I list attributes of class: article | +-----------------+--------+-----+------------+------+------+------+-----+-----------------+ I Identifier | Lang | Id | Type | Srch | Reqd | Info | Pos | Name | +-----------------+--------+-----+------------+------+------+------+-----+-----------------+ | title | eng-US | 183 | ezstring | 1 | 1 | 0 | 1 | Title | | short_title | eng-US | 184 | ezstring | 1 | 0 | 0 | 2 | Short title | | author | eng-US | 185 | ezauthor | 0 | 0 | 0 | 3 | Author | | intro | eng-US | 186 | ezxmltext | 1 | 1 | 0 | 4 | Summary | | body | eng-US | 187 | ezxmltext | 1 | 0 | 0 | 5 | Body | | enable_comments | eng-US | 188 | ezboolean | 0 | 0 | 0 | 6 | Enable comments | | image | eng-US | 189 | ezimage | 0 | 0 | 0 | 7 | Image | | caption | eng-US | 190 | ezxmltext | 1 | 0 | 0 | 8 | Caption (Image) | | publish_date | eng-US | 191 | ezdatetime | 1 | 0 | 0 | 9 | Publish date | | unpublish_date | eng-US | 192 | ezdatetime | 1 | 0 | 0 | 10 | Unpublish date | | tags | eng-US | 193 | ezkeyword | 1 | 0 | 0 | 11 | Tags | | star_rating | eng-US | 194 | ezsrrating | 0 | 0 | 0 | 12 | Star Rating | +-----------------+--------+-----+------------+------+------+------+-----+-----------------+
It could only be 'star rating'. What is the status of the extension:
> eep list extensions +--------------------+--------------------+----------------+---------------+---------------+ I list extensions | +--------------------+--------------------+----------------+---------------+---------------+ I folders | ActiveExtensions | A.A.Extensions | design | modules | +--------------------+--------------------+----------------+---------------+---------------+ | contentdeployment | | | | | | ezjscore | ezjscore | | ezjscore | ezjscore | | ezodf | | | | | | ezcomments | | | | | | ezscriptmonitor | | | | | | ezsi | | | | | | ezie | | | | | | data_import | | | | | | ezmultiupload | ezmultiupload | | ezmultiupload | ezmultiupload | | ezstarrating | | | | | | dbx | dbx | | dbx | dbx | | ezfind | ezfind | | ezfind | ezfind | | ezwebin | ezwebin | | ezwebin | | | ezflow | ezflow | | ezflow | ezflow | | ezmbpaex | | | | | | ezformtoken | | | | | | ezoe | ezoe | | ezoe | ezoe | | ezwt | | | | | | ezprestapiprovider | ezprestapiprovider | | | | | ezless | | | | | | ezgmaplocation | | | | | | rautils | | | | | +--------------------+--------------------+----------------+---------------+---------------+
So, the "ezstarrating" extension is not enabled. Let me hack that into settings/override/site.ini.append.php.
> eep list extensions | grep star | ezstarrating | ezstarrating | | ezstarrating | |
And then try to delete the content class:
> eep contentclass deleteclass article Deleting 0 objects. Done deleting objects.
and then delete all the useless content classes again, and confirm:
> eep list contentclasses | awk '$6==0 {print $2}' | wc -l
0
Ok, ok; using wc was a little bit unnecessary, but it did generate a nice "0".
Example 2: Handle biggish installations
> eep list contentclasses +---------------------+----+--------+----------------------------------+--------+---------------------+-----------+ I list content classes | +---------------------+----+--------+----------------------------------+--------+---------------------+-----------+ I Identifier | Id | # | RemoteID | Lang | Name | Group | +---------------------+----+--------+----------------------------------+--------+---------------------+-----------+ | onix_bisac_category | 55 | 2733 | 85d41773c35f29bcbe41aea2eaec93c9 | eng-CA | BISAC category | Bookshelf | | blog | 19 | 1 | 3a6f9c1f075b3bf49d7345576b196fe8 | eng-CA | Blog | Content | | blog_post | 20 | 6 | 7ecb961056b7cbb30f22a91357e0a007 | eng-CA | Blog post | Content | | book_annotation | 71 | 671 | f29e51311e8aa9821ef98e78f6c00a4b | eng-CA | Book Annotation | Bookshelf | | book_folder | 49 | 27 | acaee61d5596130a584cae3d91ee53d3 | eng-CA | Book Folder | Bookshelf | | common_ini_settings | 14 | 1 | ffedf2e73b1ea0c3e630e42e2db9c900 | eng-CA | Common ini settings | Setup | | contributor_folder | 50 | 26 | 80ec2434df8e66d72baa99f1a1079733 | eng-CA | Contributor Folder | Bookshelf | | documentation_page | 24 | 12 | d4a05eed0402e4d70fedfda2023f1aa2 | eng-CA | Documentation page | Content | | file | 28 | 4 | 637d58bfddf164627bdfd265733280a0 | eng-CA | File | Media | | flash_recorder | 30 | 1 | e349c947fd306299418be35b07b9a940 | eng-CA | Flash recorder | Media | | folder | 1 | 302 | a3d405b81be900468eb153d774f4f0d2 | eng-CA | Folder | Content | | frontpage | 23 | 6 | e36c458e3e4a81298a0945f53a2c81f4 | eng-CA | Frontpage | Content | | game_cover | 73 | 48 | 3101958361cf3b76b9c804ccc0aab1b8 | eng-CA | Game Cover | Bookshelf | | giveaway | 76 | 34 | 00edae306718c75cabbfcb738a6c0df1 | eng-CA | Giveaway | Bookshelf | | global_layout | 32 | 1 | f0271811b913befa8f062527e909f15e | eng-CA | Global layout | Content | | image | 33 | 918 | f6df12aa74e36230eb675f364fccd25a | eng-CA | Image | Media | | list | 65 | 332 | faa59e289d6d7fb6636ea2d33fc92f41 | eng-CA | List | Bookshelf | | list_bisac_category | 75 | 851 | 6ca9b6c4ac241ff77d2f9cfe0566c17c | eng-CA | List BISAC category | Bookshelf | | message | 67 | 34 | 1db77616a9436fdb19d8d836c66dd8db | eng-CA | Message | Bookshelf | | otp_post | 64 | 178 | 1bd7e59a75b52e276b0755aa62e089db | eng-CA | Off The Page post | Bookshelf | | onix_contributor | 54 | 27524 | 6885e3e3d924dd57e12019a56a7fa716 | eng-CA | ONIX Contributor | Bookshelf | | onix_mediafile | 56 | 37415 | 658867ac3bd41a95e71eee070b0542e5 | eng-CA | ONIX MediaFile | Bookshelf | | onix_othertext | 57 | 134714 | 9daf9eb9ff4f79f51ac58bd8441f3562 | eng-CA | ONIX OtherText | Bookshelf | | onix_prize | 62 | 9481 | d37435393a5d53231e188edeb1b93f1d | eng-CA | ONIX Prize | Bookshelf | | onix_product | 52 | 52364 | fdbb7e3150f22c72f5fadb8963ffc045 | eng-CA | ONIX Product | Bookshelf | | onix_publisher | 53 | 745 | d0287045337b0f2fc9c9e03d188f23e6 | eng-CA | ONIX Publisher | Bookshelf | | onix_series | 61 | 1961 | d492b9b7a14b8e0e64ad129270f67f7c | eng-CA | ONIX Series | Bookshelf | | onix_supplydetail | 59 | 53704 | f91869f0f114a08b8b3a1842709fc665 | eng-CA | ONIX Supply Detail | Bookshelf | | onix_website | 58 | 4841 | b5acae96ec526d0bdb0c0d7996b1ffd7 | eng-CA | ONIX Website | Bookshelf | | publisher_folder | 51 | 27 | e7f78d2f9c8a7bb10d5afc57092dc0c7 | eng-CA | Publisher Folder | Bookshelf | | report_message | 72 | 274 | 64eeb2569da631066cb024b85a812af2 | eng-CA | Report Message | Bookshelf | | review | 70 | 95 | 3331aaebb0ca37e1834d0bcc3ccabfcf | eng-CA | Review | Bookshelf | | review_folder | 69 | 86 | 63b33166599d4b499b62ac76b12a89c5 | eng-CA | Review Folder | Bookshelf | | series_folder | 60 | 27 | 5b140326284981ede98b9bb6d1ed3cba | eng-CA | Series Folder | Bookshelf | | simple_content | 74 | 2 | 690aa81fdb1ca8b18a4f84a37f2e2ccb | eng-CA | Simple content | Bookshelf | | template_look | 15 | 1 | 59b43cd9feaaf0e45ac974fb4bbd3f92 | eng-CA | Template look | Setup | | user | 4 | 1398 | 40faa822edc579b02c25f6bb7beec3ad | eng-CA | User | Users | | user_affiliation | 63 | 4 | 91f4a128ca662425698c314ac38fc1b0 | eng-CA | User Affiliation | Bookshelf | | user_group | 3 | 6 | 25b4268cdcd01921b808a0d854b877ef | eng-CA | User group | Users | +---------------------+----+--------+----------------------------------+--------+---------------------+-----------+
That's 134714 instances of onix_othertext. Handling this many instances in PHP is something of a problem because memory leaks will crash your script well before you've handled all those objects.
> eep contentclass fetchallinstances onix_othertext
PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 71 bytes) in /home/dfp/eep/lib/eepHelpers.php on line 352
PHP Stack trace:
PHP 1. {main}() /home/dfp/eep/eep.php:0
PHP 2. require_once() /home/dfp/eep/eep.php:130
PHP 3. contentclass_commands->run() /home/dfp/eep/modules/contentclass/index.php:311
PHP 4. contentclass_commands->fetchallinstances() /home/dfp/eep/modules/contentclass/index.php:265
PHP 5. eep::displayNonObjectList() /home/dfp/eep/modules/contentclass/index.php:171
Fatal error: eZ Publish did not finish its request
The execution of eZ Publish was abruptly ended, the debug output is present below.
So eep supports limit and offset:
> eep contentclass fetchallinstances onix_othertext --limit=100000 --offset=40000 +---------+-----+---+----------------------------------+---------------------------------------------+ I All instances of content class 'onix_othertext' | +---------+-----+---+----------------------------------+---------------------------------------------+ I Object | Sid | V | Remote Id | Name | +---------+-----+---+----------------------------------+---------------------------------------------+ | 3044426 | 1 | 1 | b5a96b469819f9f752bf8f4118ff25ba | Main description | | 3044427 | 1 | 1 | b67005432cac1b0afd231f60c1a6ba2c | Review text | | 3044431 | 1 | 1 | 6e57bd7c41b752e9bca9b085ce0d1281 | Main description | | 3044435 | 1 | 1 | 8e88848541395e344384f2b65e67f3e0 | Main description | | 3044439 | 1 | 1 | de8550226af4248bd951a775acee4545 | Main description |
... and so on, generating 100,000 rows of data.
Example 3: Managing data
eep wraps the lovely toString() and fromString() API functions. This is the built-in 'help' output:
> eep attribute Available commands:: help, delete, fromstring, tostring, migrate, newattributexml, update delete - deletes an attribute from class and objects eep use ezroot <path> eep use contentclass <class identifier> eep attribute delete <attribute identifier> fromstring - calls FromString() on the attribute eep use contentobject <object id> eep attribute fromstring <attribute identifier> <new value> or eep attribute fromstring <content object id> <attribute identifier> <new value> tostring - calls ToString on the attribute eep use contentobject <object id> eep attribute tostring <attribute identifier> or eep attribute tostring <content object id> <attribute identifier> migrate - copies data from one attribute to another within a content class - todo, report available conversions currently supported are "rot13" for testing and "time2integer" eep use ezroot <path> eep use contentclass <class identifier> eep attribute migrate <src attribute> <conversion> <dest attribute> newattributexml - dumps xml that can be edited and then imported eep attribute newattributexml update - updates objects with new attribute and also the class; will resume after a partial update eep use ezroot <path> eep use contentclass <class identifier> eep attribute update <path to newattributexml file>
Download and contribute to eep on GitHub: https://github.com/mugoweb/eep
