System description

DVelum -  PROFESSIONAL WEB-DEVELOPMENT PLATFORM.  PHP + ExtJS.

DVelum is a quick development platform based on ExtJS framework and PHP + MySQL on the server side. The platform opportunities allow to create a ready-to-go application in minutes without using complex XML configuration files, all settings being adjusted in visual interfaces. Automatically generated interfaces are easy to modify with the help of built-in layout designer.

The platform can be applied for rapid prototyping as well as for creating business applications, back offices and data management interfaces.

The main system components are:

  • lightweight ORM with visual management interface;
  • code auto generator with editable auto generating templates;
  • layout designer with built-in code editor.

Besides, the platform features an integrated PSR-0 compatible class auto loader that allows ready-made functionality and libraries to be easily integrated.

Above all, DVelum offers the highest performance and flexible settings along with a wide range of components and opportunities facilitating the development process.

This is not another cms or framework, it is a platform for the development of systems with a complex internal structure, such as eCommerce, CRM, WebScada, etc., while the development of simple sites with its help is as easy and pleasant.

BASIC PROJECT CONCEPTS:

  • maximum automation of the development process;
  • use of visual interfaces for development purposes.
  • maximum performance;

SYSTEM REQUIREMENTS

  • Linux (trying to run the installation file in Windows may cause potential compatibility problems when operating the software, we are working on compatibility issues);
  • PHP 5.3 and higher;
  • Mysql 5.1 or higher;
  • Apache web server + mod_rewrite / Nginx;
  • memcached and opcode cache are recommended.

The software package consists of libraries, applications and web interfaces to manage them.

Platform operates in two modes:

  • production system;
  • development system.

Currently, the platform has three interfaces:

  • public section;
  • administrative interface;
  • developer interface.

The operation modes differ in the functionality set and performance configuration.

There is supposed to be an option to change the interface language.

FUNCTIONALITY SET

EXTJS ADMINISTRATION PANEL

Developer mode:

System Administrator Mode:

PUBLIC PORTAL COMPONENT

  • pages;
  • news;
  • block structure;
  • theming;
  • blocks for various modules (news, pages, articles, menus, static, blocks, topics).

The public portal component includes a small set of ready-to-use solutions, which serve as examples of how the system works.

CREATING AN APPLICATION: THE WORKFLOW

  • Using the ORM management visual interface [link to ORM], create database objects, define their properties (fields) and relations.
  • Using the code auto generator located in the administrative module management section, generate the necessary controllers and interface projects (in one click).
  • Using the visual editor, improve the visual interface for managing the created objects (if required). Using the same editor, add the missing business logic for your application (JS+ExtJS). Create more controllers and models if necessary.
  • Assign access permissions in the User management interface.
  • Create the public side of your application, update its design For the public component, there are a few ready management interfaces.

SYSTEM FEATURES

ADMIN PANEL USING EXTJS4

ExtJS framework - a flexible and powerful tool for developing interfaces.

If you are not familiar with ExtJS, it is high time you started studying it.

Introduction: http://www.sencha.com/products/extjs/

Widgets review: http://www.sencha.com/products/extjs/examples/

Learning resources: http://www.sencha.com/learn/extjs/?4x

Documentation: http://docs.sencha.com/ext-js/4-0/

BASIC CACHE SYSTEM

The platform has been designed considering the necessity of data caching.

Cache can be subdivided into the necessary number of types. Each type has its own storage, which can work with various backend types. Currently, out of the box package supports cache in memcached and APC.

PSR-0 AUTOLOADER WITH CLASS MAP SUPPORT, CODE PACKAGES, CODE COMPILER

The bundle autoloader + code compiler + class map + code packages considerably speeds up classes loading and overall system performance. Autoloader uses class maps and packages (automatically generated) to find and load classes. Packages represent files with the collected source code of the related classes.

Code compiler allows you to create Code Packages (very important during the system growth).Developer determines what libraries and system classes are used continuously or in a group, and based on this, creates code packages containing sets of classes. A single click compiles packages from the source code, creating the class map for Autoloader at the same time. No need to write your own scripts that collect all the code in one file as it is not always appropriate. Downloading only the program code you actually need substantially reduces the number of costly ‘require’ operations and allows to save on RAM and CPU time.

When you request a class, Autoloader is looking for it through the map (hash) in the first place, defines its location in the file system and relation to whatever Package. If the class is included in the package the entire package gets connected, otherwise, just a separate file.

If the class is not found in the map (for example, it is outdated), Autoloader starts searching for the class in the file system by the defined paths.

Such an approach allows to increase the system performance by many times, bringing the excess of the uploaded code to the minimum.

Below goes the history of system performance tests run on the Blog functionality (we tested accessing Blog's main page containing 10 entries) and the Main page with Memcached in different DVelum versions.
 

In the process of development, we place great emphasis on tracing and profiling the code in order to detect "slow" structures and optimize them.

ORM

ORM system includes an intuitive visual control interface. No need to write a configuration file objects, everything is done by means of the interface.

What is so interesting about ORM?

  • a convenient control interface;
  • the ability to analyze table structures, import a table from another database or project;
  • the opportunity to make backup copies of the structure and data (available via localhost so far);
  • objects configuration is stored in files, so you can easily move them from server to server;
  • if the structure of an object is different from the structure of its relational representation, the system will warn you of a non-compliance issue;
  • the relational representation may be synchronized in one click, so it is easier to keep in the same SVN;
  • supports multiple MySQL storage engines like memory, myisam, innodb and, therefore, processes object data updates in different ways (automatically uses transactions when they become available);
  • validates data before an update, supports objects links and lists of objects, as well as list dictionaries (not to be confused with ENUM);
  • allows you to create indices;
  • supports data version control mechanism, which is integrated into the system;
  • supports logging of changes;
  • allows to add your own data validators;
  • referential integrity is validated independently of Storage Engine;
  • multilingual names (names can be defined for each language);
  • Foreigen Keys support;
  • allows to connect third-party database tables, import object structures.

The ORM concept is quite simple. It uses Concrete Table Inheritance approach, so, the table contains all the properties of the described class. We consciously avoided implementing ORM object inheritance in favor of simplicity, performance, and prospects for the introduction of horizontal sharding. Other approaches, if not eliminate, then greatly complicate scaling. The architectural concept is radically different from that of other systems such as Doctrine.

TWO LEVELS OF DATA PROCESSING:

  • creation, modification, deletion of data - via ORM;
  • building queries - through models.

The combination of two approaches helps to reduce the drawbacks of data processing by means of ORM. On the one hand, the increased query performance brings about an opportunity to introduce transparent data caching and invalidate cache. On the other hand, the above mentioned combination of approaches ensures integrity and validity of data.

Access to all objects is defined by the class Db_Object


$news = new Db_Object('news');
$news->title = 'First news';
$news->text = 'Some text';
$news->news_date = date('Y-m-d');
$news->save();

We would like to draw your attention to the fact that the above mechanism is not just about defining some object properties. The number and types of properties are rigidly restricted by the ORM management interface.

The main thing, which makes it different from Doctrine, is that there is no need to write PHP-code for objects description or to inherit objects from base classes. Consequently,  Db_Object can not implement private behavior of an object, the behaviour being described by the model or an additional class. Object relations are defined by means of the interface.

ORM DYNAMIC MODELS

In the absence of a Model for a particular object, there is instantiated a virtual model, which uses the basic functionality of data processing (such as retrieving a list of objects by matching a condition, by filtering or by a search request; counting the number of objects, pagination, etc.). There is also an opportunity to define the real classes inherited from Model and expanding its functionality. See more in the section "ORM and Models".

INTERACTIVE UML-DIAGRAM OF ORM OBJECT RELATIONS

The system builds an interactive chart of ORM object relations, which will help you to quickly figure out the data structure.

CODE MANAGEMENT, CONNECTING JS,  COMPILATION, CACHING

JS-code management system includes compilation, caching and output.

VISUAL REPORT DESIGNER

The graphical report interface will perform analysis on the data collected, generate and export reports.

See more in the section “Description of Reports”.

MEDIA LIBRARY

Media Library - a storage for multimedia content and files, easy search, download/multi-download and editing, automatic and manual cutting.

An image is automatically cut during file upload, you can specify the required number and size of thumbnails, as well as the method of cutting. It is possible to simply reduce an image or fit it to a specific size constraining the proportions. Uploaded files are arranged into folders, thumbnail links get a postfix according to the configuration settings (we have also presupposed a mechanism for creating a link for a thumbnail of a specific size).

Media Library has two file uploaders including multiupload.

DVelum HTML5 multifile uploader

Once a media library is formed,  images can be attached to other system objects and embedded into the WYSIWYG.

Quite common are situations of web project redesign or just adding new elements of a different size that presuppose updating the size of thumbnails. 

The system  allows you to run background tasks for resizing images in different modes:

  • for all images;
  • for images that have been manually cut with the help of the crop tool;
  • images of a certain size.

The running process is displayed in the admin interface and may suspended, stopped, started with no risk of reloading the page. See more in the section "Media library section".

BACKGROUND TASKS

Background tasks are performed interactively, the task progress and its main parameters being displayed in the admin interface. Tasks can suspended, stopped, started or signaled. Reloading the page does not affect the execution of the task.

So far, we have applied the simplest approach - start by sending an AJAX request to the server in the ‘ignore_user_abort mode. Later on, we’ll implement the natural fork of the process using PCNTL.

DATA VERSION CONTROL

Data version control system is integrated into the system and may be connected for certain ORM objects.

Objects under data version control:

  • have additional system fields;
  • can be published (the published version is tracked);
  • may be reverted to the previous version;
  • allow for viewing the versions.

The changes will be displayed in the public frontend only after an article, a web-page or a document have been actually published. Links to objects and object lists are also involved in data version control. The interface displays the status of the related objects.

Prior to being published, the data may be previewed in the public interface within the current theme design.

GRANTING PERMISSIONS

Like in many other systems, there is a mechanism for granting permissions to a user / user groups for accessing different areas of the system (view, edit, delete and publish rights). See more in the section "Users section".

MANAGEMENT OF THE TREE STRUCTURE IN THE SYSTEM PUBLIC AREA

The toolkit may be conveniently used for drag&dropping the elements of the tree structure, performing data version control, previewing pages or managing blocks. See more in the section "Pages section".

MANAGING BLOCKS

The interface allows for drag&dropping block elements within various page layouts and an unlimited number of containers. The page layout is defined in the theme settings by a simple configuration file (similar to table HTML, might be replaced by a visual interface in future). The interface also allows to pick out a block structure, which thereafter is to be moderated by administrators. See more in the section "Blocks section".

Exemplary configuration file:

return array(
  'columns'=>3,
  'items'=>array(
    'b1'=>array(
      'colspan'=>3,
      'container'=>false,
      'width'=>600,
      'height'=>80,
      'html'=>'Header'
    ),
    'leftblock'=>array(
      'container'=>true,
      'width'=>150,
      'height'=>300
    ),
    'b3'=>array(
      'container'=>false,
      'dataArea'=>true,
      'width'=>300,
      'height'=>300,
      'html'=>'Main content'
    ),
    'rightblock'=>array(
      'container'=>true,
      'width'=>150,
      'height'=>300
    ),
    'b5'=>array(
      'colspan'=>3,
      'container'=>false,
      'width'=>600,
      'height'=>80,
      'html'=>'footer'
    )
  )
);

THEMES

You can specify your own theme and blocks layout for each page of the public interface.

MENU DESIGNER

The user-friendly interface helps to create and edit the tree-like structure of the menu, allowing for the unlimited amount of menus. A menu can be linked to a template or be included into a block and may appear on any page. See more in the section "Menu section".

EVENT MANAGER, TRIGGERS

The cache invalidation issue has been resolved by creating an event manager, operating more like a trigger manager. Prior to any operations, the object storage sends an event to the manager, which checks triggers availability.

Built-in IDE, Layout Designer

We are still heartily working on integrating an IDE, which will boost the efficiency of interface development. Would any professional PHP developer ever mind having a visual development environment similar to MSVisual Studio / Borland C + + builder / RAD Studio?

What kind of system is Dvelum IDE and how is it different from its counterparts?

The visual-interface allowing to design the system layout is integrated into the system, which is remarkable enough as it automates the development process even more. The functionality set of the interface may be extended to suit your application needs, which is the option the similar systems lack.  The interface is tightly integrated with ORM, it can analyze the database structure and use it to create forms and grids.

You can easily write your own add-ons and widgets.

The interface under development may be previewed and tested right in the editor window. At the current stage you are able to modify window size, grid columns’ width and position by drag&dropping them in the preview area. Drag&drop feature as well as the rest of functionalities are implemented by interacting with components of the development environment.

URL addresses are formed by the component analyzing the platform code and offering a Controller and an Action to select. The component itself forms the query path (comments to code will show up if bytecode caching is off), and does many other minor things, which might be not covered by Ext Architect.

Currently the IDE features an integrated visual JS code editor, an ability to connect external JS-files and embedded projects.

It is worth mentioning that adding a new administrative module to the project may be performed in one click (later on we will introduce the same opportunity for a client module), which considerably facilitates the development process. Click Backend Modules Section -> Create Interface to create a ready CRUD-interface, which can be changed to your liking.

Use of IDE is not at all required, if you can manage development by yourself, describing controllers and creating interface projects on your own.

Thanks to optimization practices, code generator of the current system version performs 10 times faster compared to that of the previous version and as of now a complex interface is generated in 0.2 seconds. It is worth mentioning that code is generated at first request only, the further requests taking 0.02 seconds. Using built-in cache in memcached may help to further optimize the performance (0.01s). We still continue to work on optimizing the code generator and improving the overall system performance. See more in the section "IDE Part I: Main Elements".

Scaffolding

A standard CRUD interface for object data control may be created in the admin modules manager in a couple of clicks. The controller, the necessary router and configuration records, administrative rights and a Layout Designer project are created automatically. Having created a module in one click, you no longer have to bother yourself with routine development of standard functionality and may focus on customizing the interface. See more in screencast "A simple web application created on the DVelum platform: Phone book in 10 minutes without programming (screencast)" and the section "A simple web application created on the DVelum platform: Phone book in 10 minutes without programming".

Deployment

Publication of updates to production-server is one of the most important topics, which could not have been left unnoticed. We have automated the publication process, however, the module is currently available with limited functionality, which should be expanded in future releases.

Besides building update packages, the Deploy module monitors remote servers’ status helping to identify corrupted or infected files.The module has certain system requirements, it works only under Linux and requires php_curl and php_zip libraries to be installed.

The Deploy module may be also used as a tool monitoring malicious code intrusions (I think many of you are familiar the such problems as hacked servers or malicious code injections). See more in the section "Deployment component (building and publishing update packages, monitoring)".