[ Index ]

PHP Cross Reference of Unnamed Project

title

Body

[close]

/se3-unattended/var/se3/unattended/install/linuxaux/opt/perl/lib/site_perl/5.10.0/i586-linux-thread-multi/DBI/ -> Roadmap.pm (source)

   1  =head1 NAME
   2  
   3  DBI::Roadmap - Planned Enhancements for the DBI
   4  
   5  Tim Bunce - 12th November 2004
   6  
   7  =head1 SYNOPSIS
   8  
   9  This document gives a high level overview of the future of the Perl
  10  DBI module.
  11  
  12  The DBI module is the standard database interface for Perl applications.
  13  It is used worldwide in countless applications, in every kind of
  14  business, and on platforms from clustered super-computers to PDAs.
  15  Database interface drivers are available for all common databases
  16  and many not-so-common ones.
  17  
  18  The planned enhancements cover testing, performance, high availability
  19  and load balancing, batch statements, Unicode, database portability,
  20  and more.
  21  
  22  Addressing these issues together, in coordinated way, will help
  23  ensure maximum future functionality with minimal disruptive
  24  (incompatible) upgrades.
  25  
  26  =head1 SCOPE
  27  
  28  Broad categories of changes are outlined here along with some
  29  rationale, but implementation details and minor planned enhancements
  30  are omitted.  More details can be found in:
  31  L<http://svn.perl.org/modules/dbi/trunk/ToDo>
  32  
  33  
  34  =head1 CHANGES AND ENHANCEMENTS
  35  
  36  These are grouped into categories and are not listed in any particular order.
  37  
  38  =head2 Performance
  39  
  40  The DBI has always treated performance as a priority. Some parts
  41  of the implementation, however, remain unoptimized, especially in
  42  relation to threads.
  43  
  44  * When the DBI is used with a Perl built with thread support enabled
  45  (such as for Apache mod_perl 2, and some common Linux distributions)
  46  it runs significantly slower. There are two reasons for this and
  47  both can be fixed but require non-trivial changes to both the DBI
  48  and drivers.
  49  
  50  * Connection pooling in a threaded application, such as mod_perl,
  51  is difficult because DBI handles cannot be passed between threads.
  52  An alternative mechanism for passing connections between threads
  53  has been defined, and an experimental connection pool module
  54  implemented using it, but development has stalled.
  55  
  56  * The majority of DBI handle creation code is implemented in Perl.
  57  Moving most of this to C will speed up handle creation significantly.
  58  
  59  * The popular fetchrow_hashref() method is many times slower than
  60  fetchrow_arrayref(). It has to get the names of the columns, then
  61  create and load a new hash each time. A $h->{FetchHashReuse} attribute
  62  would allow the same hash to be reused each time making fetchrow_hashref()
  63  about the same speed as fetchrow_arrayref().
  64  
  65  * Support for asynchronous (non-blocking) DBI method calls would
  66  enable applications to continue processing in parallel with database
  67  activity.  This is also relevant for GUI and other event-driven
  68  applications.  The DBI needs to define a standard interface for
  69  this so drivers can implement it in a portable way, where possible.
  70  
  71  These changes would significantly enhance the performance of the
  72  DBI and many applications which use the DBI.
  73  
  74  
  75  =head2 Testing
  76  
  77  The DBI has a test suite. Every driver has a test suite.  Each is
  78  limited in its scope.  The driver test suite is testing for behavior
  79  that the driver author I<thinks> the DBI specifies, but may be
  80  subtly incorrect.  These test suites are poorly maintained because
  81  the return on investment for a single driver is too low to provide
  82  sufficient incentive.
  83  
  84  A common test suite that can be reused by all the drivers is needed.
  85  It would:
  86  
  87  * Improve the quality of the DBI and drivers.
  88  
  89  * Ensure all drivers conform to the DBI specification.  Easing the
  90  porting of applications between databases, and the implementation
  91  of database independent modules layered over the DBI.
  92  
  93  * Improve the DBI specification by clarifying unclear issues in
  94  order to implement test cases.
  95  
  96  * Encourage expansion of the test suite as driver authors and others
  97  will be motivated by the greater benefits of their contributions.
  98  
  99  * Detect and record optional functionality that a driver has not
 100  yet implemented.
 101  
 102  * Improve the testing of DBI subclassing, DBI::PurePerl and the
 103  various "transparent" drivers, such as DBD::Proxy and DBD::Multiplex,
 104  by automatically running the test suite through them.
 105  
 106  These changes would improve the quality of all applications using
 107  the DBI.
 108  
 109  
 110  =head2 High Availability and Load Balancing
 111  
 112  * The DBD::Multiplex driver provides a framework to enable a wide
 113  range of dynamic functionality, including support for high-availability,
 114  failover, load-balancing, caching, and access to distributed data.
 115  It is currently being enhanced but development has stalled.
 116  
 117  * The DBD::Proxy module is complex and relatively inefficient because
 118  it's trying to be a complete proxy for most DBI method calls.  For
 119  many applications a simpler proxy architecture that operates with
 120  a single round-trip to the server would be simpler, faster, and more
 121  flexible.
 122  
 123  New proxy client and server classes are needed, which could be
 124  subclassed to support specific client to server transport mechanisms
 125  (such as HTTP and Spread::Queue).  Apart from the efficiency gains,
 126  this would also enable the use of a load-balanced pool of stateless
 127  servers for greater scalability and reliability.
 128  
 129  * The DBI currently offers no support for distributed transactions.
 130  The most useful elements of the standard XA distributed transaction
 131  interface standard could be included in the DBI specification.
 132  Drivers for databases which support distributed transactions could
 133  then be extended to support it.
 134  
 135  These changes would enable new kinds of DBI applications for critical
 136  environments.
 137  
 138  
 139  =head2 Unicode
 140  
 141  Use of Unicode with the DBI is growing rapidly. The DBI should do
 142  more to help drivers support Unicode and help applications work
 143  with drivers that don't yet support Unicode directly.
 144  
 145  * Define expected behavior for fetching data and binding parameters.
 146  
 147  * Provide interfaces to support Unicode issues for XS and pure Perl
 148  drivers and applications.
 149  
 150  * Provide functions for applications to help diagnose inconsistencies
 151  between byte string contents and setting of the SvUTF8 flag.
 152  
 153  These changes would smooth the transition to Unicode for many
 154  applications and drivers.
 155  
 156  
 157  =head2 Batch Statements
 158  
 159  Batch statements are a sequence of SQL statements, or a stored
 160  procedure containing a sequence of SQL statements, which can be
 161  executed as a whole.
 162  
 163  Currently the DBI has no standard interface for dealing with multiple
 164  results from batch statements.  After considerable discussion, an
 165  interface design has been agreed upon with driver authors, but has
 166  not yet been implemented.
 167  
 168  These changes would enable greater application portability between
 169  databases, and greater performance for databases that directly
 170  support batch statements.
 171  
 172  
 173  =head2 Introspection
 174  
 175  * The methods of the DBI API are installed dynamically when the DBI
 176  is loaded.  The data structure used to define the methods and their
 177  dispatch behavior should be made part of the DBI API. This would
 178  enable more flexible and correct behavior by modules subclassing
 179  the DBI and by dynamic drivers such as DBD::Proxy and DBD::Multiplex.
 180  
 181  * Handle attribute information should also be made available, for
 182  the same reasons.
 183  
 184  * Currently is it not possible to discover all the child statement
 185  handles that belong to a database handle (or all database handles
 186  that belong to a driver handle).  This makes certain tasks more
 187  difficult, especially some debugging scenarios.  A cache of weak
 188  references to child handles would solve the problem without creating
 189  reference loops.
 190  
 191  * It is often useful to know which handle attributes have been
 192  changed since the handle was created (e.g., in mod_perl where a
 193  handle needs to be reset or cloned). This will become more important
 194  as developers start exploring use of the newly added
 195  $h1->swap_inner_handle($h2) method.
 196  
 197  These changes would simplify and improve the stability of many
 198  advanced uses of the DBI.
 199  
 200  
 201  =head2 Extensibility
 202  
 203  The DBI can be extended in three main dimensions: subclassing the
 204  DBI, subclassing a driver, and callback hooks. Each has different
 205  pros and cons, each is applicable in different situations, and
 206  all need enhancing.
 207  
 208  * Subclassing the DBI is functional but not well defined and some
 209  key elements are incomplete, particularly the DbTypeSubclass mechanism
 210  (that automatically subclasses to a class tree according to the
 211  type of database being used).  It also needs more thorough testing.
 212  
 213  * Subclassing a driver is undocumented, poorly tested and very
 214  probably incomplete. However it's a powerful way to embed certain
 215  kinds of functionality 'below' applications while avoiding some of
 216  the side-effects of subclassing the DBI (especially in relation to
 217  error handling).
 218  
 219  * Callbacks are currently limited to error handling (the HandleError
 220  and HandleSetError attributes).  Providing callback hooks for more
 221  events, such as a row being fetched, would enable utility modules,
 222  for example, to modify the behavior of a handle independent of any
 223  subclassing in use.
 224  
 225  These changes would enable cleaner and more powerful integration
 226  between applications, layered modules, and the DBI.
 227  
 228  
 229  =head2 Debugability
 230  
 231  * Enabling DBI trace output at a high level of detail causes a large
 232  volume of output, much of it probably unrelated to the problem being
 233  investigated. Trace output should be controlled by the new named-topic
 234  mechanism instead of just the trace level.
 235  
 236  * Calls to XS functions (such as many DBI and driver methods) don't
 237  normally appear in the call stack.  Optionally enabling that would
 238  enable more useful diagnostics to be produced.
 239  
 240  * Integration with the Perl debugger would make it simpler to perform
 241  actions on a per-handle basis (such as breakpoint on execute,
 242  breakpoint on error).
 243  
 244  These changes would enable more rapid application development and
 245  fault finding.
 246  
 247  
 248  =head2 Database Portability
 249  
 250  * The DBI has not yet addressed the issue of portability among SQL
 251  dialects.  This is the main hurdle limiting database portability
 252  for DBI applications.
 253  
 254  The goal is I<not> to fully parse the SQL and rewrite it in a
 255  different dialect.  That's well beyond the scope of the DBI and
 256  should be left to layered modules.  A simple token rewriting mechanism
 257  for five comment styles, two quoting styles, four placeholder styles,
 258  plus the ODBC "{foo ...}" escape syntax, is sufficient to significantly
 259  raise the level of SQL portability.
 260  
 261  * Another problem area is date/time formatting.  Since version 1.41
 262  the DBI has defined a way to express that dates should be fetched
 263  in SQL standard date format (YYYY-MM-DD).  This is one example of
 264  the more general case where bind_col() needs to be called with
 265  particular attributes on all columns of a particular type.
 266  
 267  A mechanism is needed whereby an application can specify default
 268  bind_col() attributes to be applied automatically for each column
 269  type. With a single step, all DATE type columns, for example, can
 270  be set to be returned in the standard format.
 271  
 272  These changes would enable greater database portability for
 273  applications and greater functionality for layered modules.
 274  
 275  
 276  =head2 Intellectual Property
 277  
 278  * Clarify current intellectual property status, including a review
 279    of past contributions to ensure the DBI is unemcumbered.
 280  
 281  * Establish a procedure for vetting future contributions for any
 282    intellectual property issues.
 283  
 284  These changes are important for companies taking a formal approach
 285  to assessing their risks in using Open Source software.
 286  
 287  
 288  =head2 Other Enhancements
 289  
 290  * Reduce the work needed to create new database interface drivers.
 291  
 292  * Definition of an interface to support scrollable cursors.
 293  
 294  
 295  =head2 Parrot and Perl 6
 296  
 297  The current DBI implementation in C code is unlikely to run on Perl 6.
 298  Perl 6 will target the Parrot virtual machine and so the internal
 299  architecture will be radically different from Perl 5.
 300  
 301  One of the goals of the Parrot project is to be a platform for many
 302  dynamic languages (including Python, PHP, Ruby, etc) and to enable
 303  those languages to reuse each others modules. A database interface
 304  for Parrot is also a database interface for any and all languages
 305  that run on Parrot.
 306  
 307  The Perl DBI would make an excellent base for a Parrot database
 308  interface because it has more functionality, and is more mature and
 309  extensible, than the database interfaces of the other dynamic
 310  languages.
 311  
 312  I plan to better define the API between the DBI and the drivers and
 313  use that API as the primary API for the 'raw' Parrot database
 314  interface.  This project is known a Parrot DBDI (for "DataBase
 315  Driver Interface").  The announcement can be read in
 316  <http://groups.google.com/groups?selm=20040127225639.GF38394@dansat.data-plan.com>
 317  
 318  The bulk of the work will be translating the DBI C and Perl base
 319  class code into Parrot PIR, or a suitable language that generates
 320  PIR.  The project stalled, due to Parrot not having key functionality
 321  at the time.  That has been resolved but the project has not yet
 322  restarted.
 323  
 324  Each language targeting Parrot would implement their own small
 325  'thin' language-specific method dispatcher (a "Perl6 DBI", "Python
 326  DBI", "PHP DBI" etc) layered over the common Parrot DBDI interface
 327  and drivers.
 328  
 329  The major benefit of the DBDI project is that a much wider community
 330  of developers share the same database drivers. There would be more
 331  developers maintaining less code so the benefits of the Open Source
 332  model are magnified.
 333  
 334  
 335  =head1 PRIORITIES
 336  
 337  =head2 Transition Drivers
 338  
 339  The first priority is to make all the infrastructure changes that
 340  impact drivers and make an alpha release available for driver authors.
 341  
 342  As far as possible, the changes will be implemented in a way that
 343  enables driver authors use the same code base for DBI v1 and DBI v2.
 344  
 345  The main changes required by driver authors are:
 346  
 347  * Code changes for PERL_NO_GET_CONTEXT, plus removing PERL_POLLUTE
 348  and DBIS
 349  
 350  * Code changes in DBI/DBD interface (new way to create handles, new
 351  callbacks etc)
 352  
 353  * Common test suite infrastructure (driver-specific test base class)
 354  
 355  =head2 Transition Applications
 356  
 357  A small set of incompatible changes that may impact some applications
 358  will also be made in v2.0. See http://svn.perl.org/modules/dbi/trunk/ToDo
 359  
 360  =head2 Incremental Developments
 361  
 362  Once DBI v2.0 is available, the other enhancements can be implemented
 363  incrementally on the updated foundations. Priorities for those
 364  changes have not been set.
 365  
 366  =head2 DBI v1 
 367  
 368  DBI v1 will continue to be maintained on a separate branch for
 369  bug fixes and any enhancements that ease the transition to DBI v2.
 370  
 371  =head1 RESOURCES AND CONTRIBUTIONS
 372  
 373  See L<http://dbi.perl.org/contributing> for I<how you can help>.
 374  
 375  If your company has benefited from the DBI, please consider if
 376  it could make a donation to The Perl Foundation "DBI Development"
 377  fund at L<http://dbi.perl.org/donate> to secure future development.
 378  
 379  Alternatively, if your company would benefit from a specific new
 380  DBI feature, please consider sponsoring its development through my
 381  consulting company, Data Plan Services. Work is performed rapidly
 382  on a fixed-price payment-on-delivery basis. Contact me for details.
 383  
 384  Using such targeted financing allows you to contribute to DBI
 385  development and rapidly get something specific and directly valuable
 386  to you in return.
 387  
 388  My company also offers annual support contracts for the DBI, which
 389  provide another way to support the DBI and get something specific
 390  in return. Contact me for details.
 391  
 392  Thank you.
 393  
 394  =cut


Generated: Tue Mar 17 22:47:18 2015 Cross-referenced by PHPXref 0.7.1