[ Index ] |
PHP Cross Reference of Unnamed Project |
[Summary view] [Print] [Text view]
1 =head1 NAME 2 3 DBI::Roadmap - Planned Enhancements for the DBI 4 5 Tim Bunce - 12th November 2004 6 7 =head1 SYNOPSIS 8 9 This document gives a high level overview of the future of the Perl 10 DBI module. 11 12 The DBI module is the standard database interface for Perl applications. 13 It is used worldwide in countless applications, in every kind of 14 business, and on platforms from clustered super-computers to PDAs. 15 Database interface drivers are available for all common databases 16 and many not-so-common ones. 17 18 The planned enhancements cover testing, performance, high availability 19 and load balancing, batch statements, Unicode, database portability, 20 and more. 21 22 Addressing these issues together, in coordinated way, will help 23 ensure maximum future functionality with minimal disruptive 24 (incompatible) upgrades. 25 26 =head1 SCOPE 27 28 Broad categories of changes are outlined here along with some 29 rationale, but implementation details and minor planned enhancements 30 are omitted. More details can be found in: 31 L<http://svn.perl.org/modules/dbi/trunk/ToDo> 32 33 34 =head1 CHANGES AND ENHANCEMENTS 35 36 These are grouped into categories and are not listed in any particular order. 37 38 =head2 Performance 39 40 The DBI has always treated performance as a priority. Some parts 41 of the implementation, however, remain unoptimized, especially in 42 relation to threads. 43 44 * When the DBI is used with a Perl built with thread support enabled 45 (such as for Apache mod_perl 2, and some common Linux distributions) 46 it runs significantly slower. There are two reasons for this and 47 both can be fixed but require non-trivial changes to both the DBI 48 and drivers. 49 50 * Connection pooling in a threaded application, such as mod_perl, 51 is difficult because DBI handles cannot be passed between threads. 52 An alternative mechanism for passing connections between threads 53 has been defined, and an experimental connection pool module 54 implemented using it, but development has stalled. 55 56 * The majority of DBI handle creation code is implemented in Perl. 57 Moving most of this to C will speed up handle creation significantly. 58 59 * The popular fetchrow_hashref() method is many times slower than 60 fetchrow_arrayref(). It has to get the names of the columns, then 61 create and load a new hash each time. A $h->{FetchHashReuse} attribute 62 would allow the same hash to be reused each time making fetchrow_hashref() 63 about the same speed as fetchrow_arrayref(). 64 65 * Support for asynchronous (non-blocking) DBI method calls would 66 enable applications to continue processing in parallel with database 67 activity. This is also relevant for GUI and other event-driven 68 applications. The DBI needs to define a standard interface for 69 this so drivers can implement it in a portable way, where possible. 70 71 These changes would significantly enhance the performance of the 72 DBI and many applications which use the DBI. 73 74 75 =head2 Testing 76 77 The DBI has a test suite. Every driver has a test suite. Each is 78 limited in its scope. The driver test suite is testing for behavior 79 that the driver author I<thinks> the DBI specifies, but may be 80 subtly incorrect. These test suites are poorly maintained because 81 the return on investment for a single driver is too low to provide 82 sufficient incentive. 83 84 A common test suite that can be reused by all the drivers is needed. 85 It would: 86 87 * Improve the quality of the DBI and drivers. 88 89 * Ensure all drivers conform to the DBI specification. Easing the 90 porting of applications between databases, and the implementation 91 of database independent modules layered over the DBI. 92 93 * Improve the DBI specification by clarifying unclear issues in 94 order to implement test cases. 95 96 * Encourage expansion of the test suite as driver authors and others 97 will be motivated by the greater benefits of their contributions. 98 99 * Detect and record optional functionality that a driver has not 100 yet implemented. 101 102 * Improve the testing of DBI subclassing, DBI::PurePerl and the 103 various "transparent" drivers, such as DBD::Proxy and DBD::Multiplex, 104 by automatically running the test suite through them. 105 106 These changes would improve the quality of all applications using 107 the DBI. 108 109 110 =head2 High Availability and Load Balancing 111 112 * The DBD::Multiplex driver provides a framework to enable a wide 113 range of dynamic functionality, including support for high-availability, 114 failover, load-balancing, caching, and access to distributed data. 115 It is currently being enhanced but development has stalled. 116 117 * The DBD::Proxy module is complex and relatively inefficient because 118 it's trying to be a complete proxy for most DBI method calls. For 119 many applications a simpler proxy architecture that operates with 120 a single round-trip to the server would be simpler, faster, and more 121 flexible. 122 123 New proxy client and server classes are needed, which could be 124 subclassed to support specific client to server transport mechanisms 125 (such as HTTP and Spread::Queue). Apart from the efficiency gains, 126 this would also enable the use of a load-balanced pool of stateless 127 servers for greater scalability and reliability. 128 129 * The DBI currently offers no support for distributed transactions. 130 The most useful elements of the standard XA distributed transaction 131 interface standard could be included in the DBI specification. 132 Drivers for databases which support distributed transactions could 133 then be extended to support it. 134 135 These changes would enable new kinds of DBI applications for critical 136 environments. 137 138 139 =head2 Unicode 140 141 Use of Unicode with the DBI is growing rapidly. The DBI should do 142 more to help drivers support Unicode and help applications work 143 with drivers that don't yet support Unicode directly. 144 145 * Define expected behavior for fetching data and binding parameters. 146 147 * Provide interfaces to support Unicode issues for XS and pure Perl 148 drivers and applications. 149 150 * Provide functions for applications to help diagnose inconsistencies 151 between byte string contents and setting of the SvUTF8 flag. 152 153 These changes would smooth the transition to Unicode for many 154 applications and drivers. 155 156 157 =head2 Batch Statements 158 159 Batch statements are a sequence of SQL statements, or a stored 160 procedure containing a sequence of SQL statements, which can be 161 executed as a whole. 162 163 Currently the DBI has no standard interface for dealing with multiple 164 results from batch statements. After considerable discussion, an 165 interface design has been agreed upon with driver authors, but has 166 not yet been implemented. 167 168 These changes would enable greater application portability between 169 databases, and greater performance for databases that directly 170 support batch statements. 171 172 173 =head2 Introspection 174 175 * The methods of the DBI API are installed dynamically when the DBI 176 is loaded. The data structure used to define the methods and their 177 dispatch behavior should be made part of the DBI API. This would 178 enable more flexible and correct behavior by modules subclassing 179 the DBI and by dynamic drivers such as DBD::Proxy and DBD::Multiplex. 180 181 * Handle attribute information should also be made available, for 182 the same reasons. 183 184 * Currently is it not possible to discover all the child statement 185 handles that belong to a database handle (or all database handles 186 that belong to a driver handle). This makes certain tasks more 187 difficult, especially some debugging scenarios. A cache of weak 188 references to child handles would solve the problem without creating 189 reference loops. 190 191 * It is often useful to know which handle attributes have been 192 changed since the handle was created (e.g., in mod_perl where a 193 handle needs to be reset or cloned). This will become more important 194 as developers start exploring use of the newly added 195 $h1->swap_inner_handle($h2) method. 196 197 These changes would simplify and improve the stability of many 198 advanced uses of the DBI. 199 200 201 =head2 Extensibility 202 203 The DBI can be extended in three main dimensions: subclassing the 204 DBI, subclassing a driver, and callback hooks. Each has different 205 pros and cons, each is applicable in different situations, and 206 all need enhancing. 207 208 * Subclassing the DBI is functional but not well defined and some 209 key elements are incomplete, particularly the DbTypeSubclass mechanism 210 (that automatically subclasses to a class tree according to the 211 type of database being used). It also needs more thorough testing. 212 213 * Subclassing a driver is undocumented, poorly tested and very 214 probably incomplete. However it's a powerful way to embed certain 215 kinds of functionality 'below' applications while avoiding some of 216 the side-effects of subclassing the DBI (especially in relation to 217 error handling). 218 219 * Callbacks are currently limited to error handling (the HandleError 220 and HandleSetError attributes). Providing callback hooks for more 221 events, such as a row being fetched, would enable utility modules, 222 for example, to modify the behavior of a handle independent of any 223 subclassing in use. 224 225 These changes would enable cleaner and more powerful integration 226 between applications, layered modules, and the DBI. 227 228 229 =head2 Debugability 230 231 * Enabling DBI trace output at a high level of detail causes a large 232 volume of output, much of it probably unrelated to the problem being 233 investigated. Trace output should be controlled by the new named-topic 234 mechanism instead of just the trace level. 235 236 * Calls to XS functions (such as many DBI and driver methods) don't 237 normally appear in the call stack. Optionally enabling that would 238 enable more useful diagnostics to be produced. 239 240 * Integration with the Perl debugger would make it simpler to perform 241 actions on a per-handle basis (such as breakpoint on execute, 242 breakpoint on error). 243 244 These changes would enable more rapid application development and 245 fault finding. 246 247 248 =head2 Database Portability 249 250 * The DBI has not yet addressed the issue of portability among SQL 251 dialects. This is the main hurdle limiting database portability 252 for DBI applications. 253 254 The goal is I<not> to fully parse the SQL and rewrite it in a 255 different dialect. That's well beyond the scope of the DBI and 256 should be left to layered modules. A simple token rewriting mechanism 257 for five comment styles, two quoting styles, four placeholder styles, 258 plus the ODBC "{foo ...}" escape syntax, is sufficient to significantly 259 raise the level of SQL portability. 260 261 * Another problem area is date/time formatting. Since version 1.41 262 the DBI has defined a way to express that dates should be fetched 263 in SQL standard date format (YYYY-MM-DD). This is one example of 264 the more general case where bind_col() needs to be called with 265 particular attributes on all columns of a particular type. 266 267 A mechanism is needed whereby an application can specify default 268 bind_col() attributes to be applied automatically for each column 269 type. With a single step, all DATE type columns, for example, can 270 be set to be returned in the standard format. 271 272 These changes would enable greater database portability for 273 applications and greater functionality for layered modules. 274 275 276 =head2 Intellectual Property 277 278 * Clarify current intellectual property status, including a review 279 of past contributions to ensure the DBI is unemcumbered. 280 281 * Establish a procedure for vetting future contributions for any 282 intellectual property issues. 283 284 These changes are important for companies taking a formal approach 285 to assessing their risks in using Open Source software. 286 287 288 =head2 Other Enhancements 289 290 * Reduce the work needed to create new database interface drivers. 291 292 * Definition of an interface to support scrollable cursors. 293 294 295 =head2 Parrot and Perl 6 296 297 The current DBI implementation in C code is unlikely to run on Perl 6. 298 Perl 6 will target the Parrot virtual machine and so the internal 299 architecture will be radically different from Perl 5. 300 301 One of the goals of the Parrot project is to be a platform for many 302 dynamic languages (including Python, PHP, Ruby, etc) and to enable 303 those languages to reuse each others modules. A database interface 304 for Parrot is also a database interface for any and all languages 305 that run on Parrot. 306 307 The Perl DBI would make an excellent base for a Parrot database 308 interface because it has more functionality, and is more mature and 309 extensible, than the database interfaces of the other dynamic 310 languages. 311 312 I plan to better define the API between the DBI and the drivers and 313 use that API as the primary API for the 'raw' Parrot database 314 interface. This project is known a Parrot DBDI (for "DataBase 315 Driver Interface"). The announcement can be read in 316 <http://groups.google.com/groups?selm=20040127225639.GF38394@dansat.data-plan.com> 317 318 The bulk of the work will be translating the DBI C and Perl base 319 class code into Parrot PIR, or a suitable language that generates 320 PIR. The project stalled, due to Parrot not having key functionality 321 at the time. That has been resolved but the project has not yet 322 restarted. 323 324 Each language targeting Parrot would implement their own small 325 'thin' language-specific method dispatcher (a "Perl6 DBI", "Python 326 DBI", "PHP DBI" etc) layered over the common Parrot DBDI interface 327 and drivers. 328 329 The major benefit of the DBDI project is that a much wider community 330 of developers share the same database drivers. There would be more 331 developers maintaining less code so the benefits of the Open Source 332 model are magnified. 333 334 335 =head1 PRIORITIES 336 337 =head2 Transition Drivers 338 339 The first priority is to make all the infrastructure changes that 340 impact drivers and make an alpha release available for driver authors. 341 342 As far as possible, the changes will be implemented in a way that 343 enables driver authors use the same code base for DBI v1 and DBI v2. 344 345 The main changes required by driver authors are: 346 347 * Code changes for PERL_NO_GET_CONTEXT, plus removing PERL_POLLUTE 348 and DBIS 349 350 * Code changes in DBI/DBD interface (new way to create handles, new 351 callbacks etc) 352 353 * Common test suite infrastructure (driver-specific test base class) 354 355 =head2 Transition Applications 356 357 A small set of incompatible changes that may impact some applications 358 will also be made in v2.0. See http://svn.perl.org/modules/dbi/trunk/ToDo 359 360 =head2 Incremental Developments 361 362 Once DBI v2.0 is available, the other enhancements can be implemented 363 incrementally on the updated foundations. Priorities for those 364 changes have not been set. 365 366 =head2 DBI v1 367 368 DBI v1 will continue to be maintained on a separate branch for 369 bug fixes and any enhancements that ease the transition to DBI v2. 370 371 =head1 RESOURCES AND CONTRIBUTIONS 372 373 See L<http://dbi.perl.org/contributing> for I<how you can help>. 374 375 If your company has benefited from the DBI, please consider if 376 it could make a donation to The Perl Foundation "DBI Development" 377 fund at L<http://dbi.perl.org/donate> to secure future development. 378 379 Alternatively, if your company would benefit from a specific new 380 DBI feature, please consider sponsoring its development through my 381 consulting company, Data Plan Services. Work is performed rapidly 382 on a fixed-price payment-on-delivery basis. Contact me for details. 383 384 Using such targeted financing allows you to contribute to DBI 385 development and rapidly get something specific and directly valuable 386 to you in return. 387 388 My company also offers annual support contracts for the DBI, which 389 provide another way to support the DBI and get something specific 390 in return. Contact me for details. 391 392 Thank you. 393 394 =cut
title
Description
Body
title
Description
Body
title
Description
Body
title
Body
Generated: Tue Mar 17 22:47:18 2015 | Cross-referenced by PHPXref 0.7.1 |