On an unrelated point, I enjoyed the Steve Yegge blog post that I referenced earlier. There’s a lot of good stuff referenced there, and not just for users of dynamic languages. He cites work on, for example, how to make virtual method calls (nearly) as fast as static method calls, it least in places where it counts. From a paper published in 1994.

Share This | Email this page to a friend

Share This | Email this page to a friend

Enabling Client-Side Caching of Generated Content in ASP.NET.

Share This | Email this page to a friend

[20A635EB]{coreide100.bpl} DocModul.AnyModuleModified (Line 3466, "DocModul.pas" + 10) + $41
[20A635E5]{coreide100.bpl} DocModul.AnyModuleModified (Line 3466, "DocModul.pas" + 10) + $3B
[00419D8B]{bds.exe} AppMain.TAppBuilder.FileSaveAllActionUpdate (Line 4067, "ui\AppMain.pas" + 1) + $0
[200401A7]{rtl100.bpl } Classes.TBasicAction.SetOnExecute (Line 11109, "common\Classes.pas" + 9) + $9
[201513B1]{vcl100.bpl } Forms..TScrollBox + $231
[2004009D]{rtl100.bpl } Classes.TBasicAction.Destroy (Line 11048, "common\Classes.pas" + 2) + $5
[2013D8C7]{vcl100.bpl } Controls.TControl.ScaleConstraints (Line 4033, "Controls.pas" + 12) + $7
[2015E854]{vcl100.bpl } Forms.DoPosition (Line 6778, "Forms.pas" + 44) + $2
[2015E86E]{vcl100.bpl } Forms.DoPosition (Line 6778, "Forms.pas" + 44) + $1C
[2015E8DC]{vcl100.bpl } Forms.DoAlign (Line 6795, "Forms.pas" + 0) + $4
[201631AA]{vcl100.bpl } Clipbrd.TClipboard.AssignPicture (Line 386, "Clipbrd.pas" + 7) + $14
[201632F9]{vcl100.bpl } Clipbrd.TClipboard.SetAsHandle (Line 430, "Clipbrd.pas" + 0) + $9
[20162637]{vcl100.bpl } Forms.TCustomFormHelper.WriteGlassFrameBottom (Line 8945, "Forms.pas" + 2) + $0
[2016291F]{vcl100.bpl } Forms.TApplicationHelper.GetEnumAllWindowsOnActivateHint (Line 9087, "Forms.pas" + 0) + $3
[0042297A]{bds.exe } bds.bds (Line 195, "" + 7) + $7

Note the line in bold. Because this is an OnUpdate event, it’s going to repeat forever. The user is trapped. There is no chance to save your work, or to exit the IDE normally. You have to end task. Sure, this is a secondary error; it’s almost certainly the consequence of the preceding access violation, rather than a defect in the OnUpdate event itself. But because that event is not robust enough to handle the consequences of the previous error, the user is trapped.

My conclusion is that OnUpdate event handlers are dangerous if they do not detect anything that could possibly go wrong with the references they use, and exit gracefully in that case.

Share This | Email this page to a friend

Share This | Email this page to a friend

DocAdapter is accessible in two ways: as a set of web services and as a set of .NET assemblies/Delphi packages. The web services and assemblies reference each other (the web service calls the assemblies to perform its tasks, and the assemblies can reference the web service to perform their tasks remotely). Because the web service and the assemblies know each other, we don’t have to worry about type matching (making sure the types used by the web service match those used by the assemblies).

Client applications can use either technology, or both. GetPublished, for example, uses the DocAdapter web service to perform the conversion from the source format to XHTML, and the CDN.Documents assembly to generate HTML, thumbnails, and other document elements.

Format Conversion

The CDN.DocumentConverters assembly (and its web service wrapper, cleverly titled DocAdapterService) contains conversion classes capable of reading files in several formats. The classes convert document text to XHTML, which is a convenient format for additional processing. Depending on the format, they can also extract additional data. For example, the Word conversion class extracts embedded images from Word documents, stores them as separate files, and creates <img> elements in the XHTML that refer to these files. A conversion class is any implementation of the IDocumentImport interface:

IDocumentImport = interface
  procedure Import(inputStream: Stream; docAdapter: DocumentAdapter;
    extractFields: Boolean);
end;

Because we’re using an interface, the DocumentAdapter class doesn’t need to know anything about the conversion class other than the fact it implements the interface. This means we can implement converters without recompiling the CDN.Documents assembly, and add them as plug-ins to the calling application.

The WebServiceConversion class is a special implementation of the IDocumentImport interface that calls DocAdapter web services. All DocAdapter services are based on the same definition, expressed in WSDL. All a client application needs in order to convert a document to XHTML to to pass the URL of such a web service to the WebServiceConversion class. GetPublished stores the URLs of the DocAdapter services in the database, so new formats can be supported by simply deploying a web service and adding a single record to GetPublished’s database.

Document Processing

Once the text and images are extracted, DocAdapter can create the HTML and necessary supporting files that can be displayed on a web site. This is done by calling a single method, CreateDocumentArchive, which returns a DocumentArchive object. The DocumentArchive object contains all the necessary information, such as the final HTML (including syntax highlighting), all referenced images, thumbnails, a table of contents, keywords, and other information that may be useful. The generation of these elements is controlled by parameters passed to the CreateDocumentArchive method. Some of these parameters are:

• Maximum image width. Images wider than this value will be replaced by thumbnails that link to the full-size image.
• Maximum image height. Images taller than this value will be replaced by thumbnails that link to the full-size image.
• The width of thumbnail images generated for images wider than the maximum specified width or taller than the maximum specified height.
• Whether to keep embedded images in the final document (external image references remain unmodified).
• The depth of the table of content to generate.
• Whether to produce printer-friendly output, which doesn’t include JavaScript elements for dynamically hiding and showing images and sections.

In GetPublished, most of these parameters are associated with specific content types configurable by system administrators.

Share This | Email this page to a friend

The CodeGear web site, as well as the Developer Network, the CodeGear Support site, and several other sites all run as part of a single application. This application is an ASP.NET 2.0 application, written in Delphi using RAD Studio 2007, and running on several load-balanced Windows servers.

GetPublished is the content management system. It’s also an ASP.NET 2.0 application, written in Delphi. On the client side, we also have some JavaScript code. GetPublished does most of the heavy lifting - content management, author roles and permissions, workflow, support for multiple sites, languages, and locations, and all direct data access.

SiteCentral is the internal name of the presentation layer - the application that displays web pages when you go to any of the CodeGear sites. The application doesn’t access data directly - it uses GetPublished (in a .NET assembly) as its data access and processing layer. In addition to standard pages, such as article lists and article pages, SiteCentral includes a set of pluggable modules. A lot of stuff you see on the page, such as the table of contents, article ratings, and even actual content, is displayed by these modules, which are configurable in GetPublished by users with sufficient permissions.

Content posted to the sites is processed by Document Adapters, which are pluggable modules capable of converting content in various formats to HTML that can be placed on the site. Document adapters are also written in Delphi, and are used as .NET assemblies and as a web service.

The applications talk to other CDN services, such as the membership service to access user accounts and the digital signatures service to accept digital signatures for submitted content.

Pages use a special template page class, which lets us separate the overall look and feel from both functional and content pages. The concept is similar to ASP.NET 2.0 master pages, but the implementation is different (mostly because we wrote it before .NET 2.0 was released), and in some ways, more flexible. For example, it’s very easy for us to replace the templates without recompiling the application.

The pages also support localization, and localized strings are also managed independently of the application. In fact, we encourage community members to help in providing localized strings in their own language, and provide a tool for this. See http://dn.codegear.com/article/33873 for details.

You can find some user-oriented documentation for the sites and application in the Help area of the Developer Network. I’ll also try to post more technical information here and/or on the Developer Network from time to time.

Share This | Email this page to a friend

When we enabled XP themes for our applications, I started noticing a flashing on the screen the first time you went to each tab in the page control. A little testing demonstrated that the gradient background of the tab was being rendered before the controls were created.

This turned out to be a fairly difficult problem to solve. When the user changes tabs, I need to figure out to which tab they are changing, and create the controls for that tab if they are not already created. In the TPageControl.OnChanging event, I don’t know to which tab the user is changing. Unfortunately, both the TPageControl.OnChange and the TTabSheet.OnShow event occur after the background of the tab has already been painted. I needed a way to render the controls, if necessary, in between these events, but after the tab to which the user is changing is already known.

After some rooting around in the VCL source code, I came up with the following solution.

type
  TChangeToPageEvent = procedure (Sender: TObject; const APageIndex: Integer) of object;
  TChangeToEventPageControl = class(TPageControl)
  private
    FOnChangeToPage: TChangeToPageEvent;
    procedure DoOnChangeToPage(const APageIndex: Integer);
  protected
    procedure Change; override;
  public
    property OnChangeToPage: TChangeToPageEvent read FOnChangeToPage write FOnChangeToPage;
  end;

{ TChangeToEventPageControl }

procedure TChangeToEventPageControl.Change;
begin
  DoOnChangeToPage(TabIndex);
  inherited;
end;

procedure TChangeToEventPageControl.DoOnChangeToPage(const APageIndex: Integer);
begin
  if Assigned(FOnChangeToPage) then begin
    FOnChangeToPage(Self, APageIndex);
  end;
end;

Since I also create the page control itself dynamically, I don’t even have to install this on the component palette to make it work. Handling the OnChangeToPage event allows me to create the controls when the page to which the user is changing is known, but before the tab itself renders.

Share This | Email this page to a friend

ASP.NET 2.0 and higher, on the other hand, includes a standard membership framework which is sufficiently extensible that I’ve never seen an application which it couldn’t be adapted to fit, and, for most applications, it will work right out of the box.

The usual way to configure page access rights for the ASP.NET membership (in web.config), however, isn’t really applicable to the new MVC framework. Scott Guthrie had promised to cover this subject, but it was Rob Conery who finally delivered the goods. In short, you decorate your controller actions with attributes specifying whether the current user must be logged in, or be within a certain role, and a controller filter handles the work of validating those security assertions at runtime. Troy Goode built upon Rob’s original code, producing the ASP.Net MVC Membership Starter Kit.

The Starter Kit includes a lot of really valuable code, but very little documentation, so I’m going to write out instructions for "getting started with the Starter Kit." I’m presuming that you’re going to use the SQL membership provider in this example, although the membership framework does include support for OpenID and Windows Live authentication as well. You can, of course, use any authentication framework you care to.

1. Download the full installer from the Releases tab at CodePlex. I’d also strongly recommend getting the source code download, which is available on the same page.
2. Run the installer.
3. If you’re using Visual Studio, you want to install the templates into VS. There’s an item on your Start menu (in the "Starter Kits") folder which will do that.
4. If you are creating a brand-new site, you can start by just copying the sample project included with the download. Skip ahead to step 10.
5. If you are adding membership support to an existing project, there’s a few things you need to do. First, you need to reference the assembly you’ve just installed. One way to do this is to simply reference the StarterKits.Mvc assembly in your project. But I chose to add the source code for the StarterKits.Mvc project to my solution instead, which helps with debugging.
6. If you haven’t done so already, edit your web.config to include Forms authentication and set the Login URL, and specify a ConnectionString if you’re not using the default. I’ll include an example of this below. Note that you need to include a membership provider and a role provider. Also, if you’re not using the default database, run aspnet_regsql.exe to add the required metadata to your database.
7. You are not required to use the controllers or views supplied with the Starter Kit. The filters in the Starter Kit will work fine with any controller you care to write. However, it’s probably easiest, especially at the beginning, to use the supplied code. If you’re using Visual Studio, you can do this with the templates you installed in step 3. Right click the Controllers folder in Solution Explorer, choose Add New Item, then choose MvcMembership Controllers FormsAuthenticationController. Do the same for FormsAuthenticationAdministrationController. Right click the Views folder, and add both View templates.
8. The current release of the Starter Kit has a bug where the codebehind and codegen files are not added along with the aspx files to the FormsAuthenticationAdministration views. This will be fixed in the next release (see Troy Goode’s comment below for details), but if you encounter this bug, I’ve included a workaround in my defect report.
9. You need to add a couple of lines to Global.asax in order to register the routes for the new controllers. You can copy those from the sample project.
10. Your site should now work, but there are a few things you need to do in order to configure e-mail and the like. These are indicated with TODO comments. You can find these in the Task List.

As promised, here is what you need to add to web.config to use a custom database. I’ve omitted the connection string definition, which is specified like any other connection string.

<authentication mode="Forms">
  <forms loginUrl="/Login"/>
</authentication>
<roleManager enabled="true">
  <providers>
    <clear/>
    <add name="AspNetSqlRoleProvider" connectionStringName="MyConnectionString"
      applicationName="MyApp" type="System.Web.Security.SqlRoleProvider, System.Web,
      Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </providers>
</roleManager>
<membership defaultProvider="MySqlMembershipProvider">
  <providers>
    <add name="MySqlMembershipProvider"
      type="System.Web.Security.SqlMembershipProvider"
      connectionStringName="MyConnectionString"
      enablePasswordRetrieval="false"
      enablePasswordReset="true"
      requiresQuestionAndAnswer="true"
      applicationName="MyApp"
      requiresUniqueEmail="true"
      passwordFormat="Hashed"
      maxInvalidPasswordAttempts="5"
      minRequiredPasswordLength="7"
      minRequiredNonalphanumericCharacters="1"
      passwordAttemptWindow="10"
      passwordStrengthRegularExpression=""/>
    </providers>
</membership>

Share This | Email this page to a friend

Knuth seems uncertain as to why literate programming hasn’t really taken off:

Jon Bentley probably hit the nail on the head when he once was asked why literate programming hasn’t taken the whole world by storm. He observed that a small percentage of the world’s population is good at programming, and a small percentage is good at writing; apparently I am asking everybody to be in both subsets.

Well, sure, but even if one assumes that 1/10 of working programmers are decent at their job, and 1/10 of those can express themselves in English and a reasonably clear manner, that’s not really enough to account for the low use of literate code. There are number of good bloggers and technical publication authors, and literate coding would seem to be well suited for these tasks. Maybe we just need some better examples to get the ball rolling.

You don’t have to read it many books on software development before you’ll encounter one with source code examples which produce erroneous results, or even fail to compile altogether. It’s an absurdly common error. Taken to an extreme, it can be maddening. In the past month or so, I’ve been reading the preprints at Bruce Eckel and Jamie King’s new book on LINQ. One interesting thing about this book is that in the process of writing it, the authors have developed a build system which builds all of the source code examples, including verifying the in-line assertions, and even goes so far as to ensure that the examples which are not supposed to build at all really do fail to compile. The finished book may not be flawless, but the examples are going to work.

I think it’s a perfect application of literate programming; I cannot think of a better solution to this problem.

Share This | Email this page to a friend

Selecting your location is a Good Thing™ - when you click on the "buy" icon (the one that looks like a couple of credit cards), the site redirects you to the correct shop site for your selected location. If you’re logged in, the site uses your country selection from your CDN account, but you can override this setting by selecting a location from the menu. To go back to your account settings, select "My default location" from the menu.

The cool thing about these buttons is that they’re automatically generated based on a database of products, shop sites, and locations. Both the "download" and the "buy" buttons are generated based on where you are (physically and on the site). At least, I think it’s cool. Users don’t really case, but developers should appreciate how much cleaner this is than manually typing in links for each product page.

Anyway, getting the correct shop site is just one of many things the web site can do once you select your location. It can give you content specific to your location, for example. Other features will be made visible in the future, so make sure you set your location.

Share This | Email this page to a friend

1. Backup.sql

This is the backup script. Because the GetPublished database is quite large, and is constantly being updated, the script creates two backups - a full database backup, and a transaction log backup. The log backup contains all transactions since the last full backup, so we can minimize downtime and data loss in case we have to take the live database offline and swap to the new server. The backup is verified, and displays progress. Replace "my_db" with your database name, and "D:\backup" with your backup directory:

BACKUP DATABASE [my_db] TO  DISK = N'D:backupmy_db.bak'
  WITH  INIT ,  NOUNLOAD ,  NAME = N'my_db backup',  NOSKIP ,  STATS = 10,  NOFORMAT
DECLARE @i INT
select @i = position from msdb..backupset
where database_name='my_db' and type!='F' and
  backup_set_id=(select max(backup_set_id)
from msdb..backupset where database_name='my_db')
RESTORE VERIFYONLY
FROM   DISK = N'D:backupmy_db.bak'
WITH FILE = @i
GO

BACKUP LOG [my_db] TO  DISK = N'D:backupmy_db_log.bak'
  WITH  INIT ,  NOUNLOAD ,  NAME = N'my_db backup',  NOSKIP ,  STATS = 10,  NOFORMAT
DECLARE @i INT
select @i = position from msdb..backupset
where database_name='my_db'and type!='F' and
  backup_set_id=(select max(backup_set_id)
from msdb..backupset where database_name='my_db')
RESTORE VERIFYONLY
FROM   DISK = N'D:backupmy_db_log.bak'
WITH FILE = @i
GO

2. Backup.cmd

This batch file simply runs the backup script:

@echo Backup started at %time%
osql -S server_name -U user_name -P password -i backup.sql
@echo Backup completed at %time%

Replace server_name with your database server name, user_name with the name of a database user with sufficient permissions to backup the database, and password with that user’s password (yes, I know that was pretty obvious). The nice thing about this batch file is that you can run it remotely. This allowed us to run everything in a single batch file on the "hot swap" machine.

3. Restore.cmd

This batch file restores the database and log backups on the local machine, using Windows authentication (you can use the -S, -U, and -P parameters to provide a SQL Server user instead):

@echo Restore started at %time%
osql -E -Q "RESTORE DATABASE [my_db] FROM DISK = 'D:backupmy_db.bak'
  WITH NORECOVERY;
  RESTORE LOG [my_db]
  FROM DISK = 'D:backupmy_db_log.bak' WITH RECOVERY;"
@echo Restore completed at %time%

Note that the call to osql has to be in a single line.

Share This | Email this page to a friend

Share This | Email this page to a friend

The general idea is that we want to start with an SQL statement, like this:

SELECT
  *
FROM
  MY_TABLE;

…and turn it into a Delphi constant, like this:

const
  MySQL = ‘SELECT ‘
        + ‘  * ‘
        + ‘FROM ‘
        + ‘  MY_TABLE;’;

The first step is to paste the SQL into the code at the appropriate place. We need to type the non-repeating portion of the constant, and then paste in the SQL:

const
  MySQL = ‘SELECT
  *
FROM
  MY_TABLE;

Now, record the macro:

1. Press the End key to go to the end of the line
2. Press Ctrl+Shift+R to start recording
3. Type a space, then the closing single quote
4. Press the down arrow, then the Home key to go to the beginning of the next line
5. Type the appropriate whitespace, then a ‘+’ sign (for string concatenation), then the open single quote character
6. Press the End key to go to the end of line, to be ready for the next time the macro is played back
7. Press Ctrl+Shift+R to stop recording

To use the macro, press and hold Ctrl+Shift+P. The keyboard auto-repeat who repeatedly play back the macro, formatting the rest of the query. Don’t worry about going too far, since you can always press Ctrl+Z to undo if you do. Since the final closing single quote and semicolon are different from how the rest of the lines are formatted, just type them.

By the way, the Delphi editor works really well for editing plain SQL. If you are editing a file with the ‘. SQL’ extension, it does syntax highlighting (and printing) for SQL. You can also use macros like the one described above when creating your SQL or DDL. For example, I often use macros to turn an INSERT statement into a UPDATE statement with parameters or trigger variable references, and similarly to turn a SELECT into a CREATE VIEW statement.

Share This | Email this page to a friend

Share This | Email this page to a friend

I use macros pretty heavily. They’re handy for just about any bulk editing task. For example, I often use them when I write SQL in InterBase PLANalyzer, and then need to paste the SQL into source code. I paste in the entire query, and then use macros to handle the quoting and string concatenation needed to turn the query statement into a valid source code, spanning several lines in the editor.

Share This | Email this page to a friend

Update: "This offer expires March 31, 2008."

Share This | Email this page to a friend

possessives.  See apostrophe’s

(Why is this funny?)

I know. I’m amused easily.

Share This | Email this page to a friend

Share This | Email this page to a friend

I’ll post the solution on Monday.

Across

1. Article
4. Someone who works with you, but not for you
8. Memory
9. Camera part
10. AOL service
11. Fishy comment?
12. Division
14. Character encoding
18. Java component
21. Toss
22. "I’m ___!"
23. Apple’s address on Infinite Loop
24. Irish high king
25. It’s illegal at the polls

Down

1. Programmer’s wiki
2. Broadway musical
3. Austen novel
4. CPU clock counts
5. Period
6. Peace, to Putin
7. Game machine
13. Tech book writer
15. Stroke cause
16. Scottish island, possible birthplace of the Book of Kells
17. Long-horned mountain goat
18. Deprecated Microsoft product
19. Composer Brian
20. Adobe development framework

Share This | Email this page to a friend

Share This | Email this page to a friend

Share This | Email this page to a friend

The MVC Web UI helpers are now in the framework. But they’ve changed quite a bit. So you must:

1. Remove the assembly from your project.
2. UpdateFrom is no longer an extension method. It’s now a static method in System.Web.Mvc.BindingHelperExtensions. So you have to change anywhere you called that.
3. Just to keep you on your toes, System.Web.Mvc.BindingHelperExtensions will probably be renamed soon.
4. RedirectToAction is now in System.Web.Routing. Edit your controllers.
5. RedirectToAction now takes a RouteValueDictionary instead of an anonymous type. You can stuff your old anonymous type into the constructor for RouteValueDictionary.
6. If you have unit tests you have to update that project, too. In a big way. All that stuff Phil Haack had on his blog as a download a while back is now in the framework. IHttpContext is now HttpContextBase. I’m still figuring this stuff out. I may have to rebuild the unit tests from scratch.

I think it’s fair to say that the unit test portion of MVC is in even greater flux than the rest of the framework at present.

Share This | Email this page to a friend

Share This | Email this page to a friend

AJAX and dynamic data features cannot be mixed in the same Web site or application as the new ASP.NET MVC. You should use separate Web sites or applications.

Share This | Email this page to a friend

Rule number one is that all components you create must have the Name property set. The regular VCL doesn’t require this for components created in code, but IntraWeb uses the Component.Name as part of the HTML id, and if the id isn’t set, the JavaScript generated by IntraWeb won’t work. So always set the Name.

Another thing to note is that the default property values don’t always make sense. If a component looks different when you dynamically generated at runtime versus when you drop it on an IntraWeb form in the designer, take a close look at the text DFM. You’ll probably find non-default property values set by the designer. If you replicate this in your code, you should end up with a component looks the same at runtime.

With all that said, here’s some simple code to add tabs at runtime to an existing tab control:

uses   Graphics, IWContainerBorderOptions;

procedure TfrmUIPageGroup.ClearPageControls;
var
  iPage: integer;
begin
  for iPage := Pred(TabControl.Pages.Count) downto 0 do begin
    TObject(TabControl.Pages[iPage]).Free;
  end;
end;

procedure TMyIntraWebForm.AddSomeTabs;
var
  iPage, iTabTop, iTabHeight, iTabWidth: integer;
  NewTab: TIWTabPage;
begin
  // need to have a design-time page to copy sizing properties from
  Assert(TabControl.Pages.Count > 0);
  iTabTop := TIWTabPage(TabControl.Pages[0]).Top;
  iTabHeight := TIWTabPage(TabControl.Pages[0]).Height;
  iTabWidth := TIWTabPage(TabControl.Pages[0]).Width;
  ClearPageControls;
  for iPage := 0 to 2 do begin
    NewTab := TIWTabPage.Create(TabControl);
    NewTab.BorderOptions.NumericWidth := 0;
    NewTab.BorderOptions.BorderWidth := cbwNumeric;
    NewTab.BorderOptions.Style := cbsNone;
    NewTab.BorderOptions.Color := clNone;
    NewTab.Color := clWebWHITE;
    NewTab.Top := iTabTop;
    NewTab.Width := iTabWidth;
    NewTab.Height := iTabHeight;
    // WordPress changes the single quote to a typographic quote here, so cutting
    // and pasting the next two lines won’t work. Sorry; I haven’t figured out how
    // to stop that
    NewTab.Name := ‘DynamicPage’ + IntToStr(iPage);
    NewTab.Title :=  ‘Page ‘ + IntToStr(iPage);
    NewTab.Parent := TabControl;
    // TabOrder determines the order of tabs (really!) in this control
    // It must be set *after* Parent
    NewTab.TabOrder := iPage;
  end;
end;

Alignment may not work as expected, either. If I build on this example by creating a control, parenting it to the tab sheet, and setting the Align property to alClient, that control’s size or position doesn’t actually change.

Finally, there is the issue of layout managers. If you use TIWTemplateProcessorHTML or TIWLayoutMgrHTML, there will not be a placeholder in your template for your dynamically generated component. There is probably a way around this, but I have not fully investigated the issue yet. One possible solution would be to do the dynamic component generation inside of a custom IntraWeb control.

Share This | Email this page to a friend

Share This | Email this page to a friend

• A stack overflow (you sometimes get a message box about this, but not always)
• Data execution prevention

Raymond Chen adds another item to the list today:

• Doing ugly things to the stack

Know of anything else? Add it in comments.

Share This | Email this page to a friend

Once a script is published into source control, it cannot be changed! Once someone updates their database with an update script, they should never have to run the same script on that same database.

Now, I think that, in general, immutability is a good thing, and it goes without saying that back version since source control should be immutable. But these scripts are not backed versions of a file; they’re a list of the DDL scripts which will actually be run on the end user’s server. And there’s the rub. Let’s say that in developing feature 100, you decide that you need to add a new column to a large table:

ALTER TABLE BIG_TABLE
  ADD FEATURE_100_COLUMN MY_DOMAIN;
UPDATE BIG_TABLE
  SET FEATURE_100_COLUMN = DEFAULT_VALUE;

Naturally, you add this script to version control, with whatever method you are using to version your database metadata changes. Later, you work on feature 150, which requires another column on the same table:

ALTER TABLE BIG_TABLE
  ADD FEATURE_150_COLUMN MY_DOMAIN;
UPDATE BIG_TABLE
  SET FEATURE_150_COLUMN = DEFAULT_VALUE;

Simply concatenating these two scripts together, as you must do if the scripts are immutable, results in an update process for the end-user which takes far longer than necessary, as the table must be altered and updated twice. It would be far more efficient to merge the two scripts:

ALTER TABLE BIG_TABLE
  ADD FEATURE_100_COLUMN MY_DOMAIN,
  ADD FEATURE_150_COLUMN MY_DOMAIN;
UPDATE BIG_TABLE
  SET FEATURE_100_COLUMN = DEFAULT_VALUE,
  SET FEATURE_150_COLUMN = DEFAULT_VALUE;

This merged script should run an approximately half of the time it takes to run the two scripts separately.

And yet, as I said earlier, treating the scripts as immutable is a good idea from a source control point of view. In particular, you want to be able to extract the metadata changes used to implement a particular feature by themselves. Hence, it makes sense to use a database metadata version control strategy which treats these two goals separately, instead of trying to combine them into a single feature. It makes sense to me to store these scripts for individual features, and have a separate script used to update a customer database.

One of Scott’s goals in developing a system he advocates is to allow/require developers to work on their own copy of the development database, which they can update along with checking out the corresponding source code, instead of using a shared, centralized development database. Merging the DDL scripts from multiple feature implementations means that one can no longer update one’s personal development database to include arbitrary new features. Instead, it is necessary to start with the metadata for the last released version of the software, and then apply the tip revision of the update script. Many developers might find this inconvenient.

Like I said, the ideal system for this has yet to be invented, as far as I can see.

Share This | Email this page to a friend

When we first migrated the old BDN site to ASP.NET, one of our goals was making sure we didn’t break existing links. To achieve that goal, we used regular expressions to search for known patterns and either redirect or rewrite the requested URLs. Over time, this became part of GetPublished, and system administrators can now add redirection patterns using the web interface.

Because these patterns have to be checked for every request, we used compiled Regex instances to boost performance. This worked great in .NET 1.1, but when we moved to 2.0, CPU usage on the servers began to spike and performance dropped like a brick. Here’s why: the .NET Regex class maintains a cache of parsed expressions. In .NET 1.1, this cache is unbounded, so all expressions are cached (until you run out of memory). Unfortunately, this behavior was changed in .NET 2.0. The cache is now limited to 15 expressions (although it is possible to override this value), and regular expressions created by Regex instance methods aren’t cached at all. This entry from the BCL team blog explains why.

Removing the RegexOptions.Compiled flag made our code over 50 times faster (or, as some would rather say, using compiled expressions made the code 50 times slower). So, if you’re using regular expressions, think hard about whether compiling them makes sense for you, and if you’re moving from .NET 1.1 to 2.0, you now know about this little performance trap just waiting to kill your application.

Share This | Email this page to a friend

This may be a hard pill to swallow for so-called "C++ programmers" or "Pascal programmers" who do not know any programming languages designed after the 1970s. But change is the rule in the software industry. If you are not learning new skills, you are eventually going to be left behind. (Note that I am not claiming that these older languages are going to go away; COBOL is still in use, too, but most people don’t make that a career goal.)

As far as I can tell, programming language designers do not seem to be creating non-garbage-collected, general-purpose programming languages anymore. This trend is not restricted to "scripting languages" or languages requiring a run time. Performance-oriented languages designed as potential replacements for C++, such as D and OCaml, are garbage collected, and the forthcoming version of the C++ standard will include features explicitly intended to facilitate use of a garbage collector.

I do not know what the programming language(s) of the future will be. Steve Yegge discussed the issue at some length, and came to the same conclusion as this post regarding GC:

[Next Big Language] is garbage collected. There will always be a bunch of engineers who think that’s evil, and they’ll continue to use C++.

C++ does need to get replaced someday. It’s just horrid, and everyone knows it. However, there aren’t very many people trying to replace it, either. The only contenders I’m aware of are Objective C and the D Programming Language.

D’s a really beautiful language. By rights it should be the next C++. However, C++ programmers won’t have it because it’s garbage collected (even though it can be disabled, and even though Stroustroup [sic.] himself is now advocating adding garbage collection to C++). Walter Bright is one hell of a lot smarter than the C++ programmers who won’t look at his language, and he has demonstrated that D is as fast as or faster than C++ and nearly as expressive as Ruby or Python. It’s a secret weapon just waiting to be seized by some smart company or open-source project.

But nobody ever accuses programmers of being wise.

Perhaps, some years from now, someone will invent a non-garbage-collected language which will become the dominant language of future days. But at the present time, I don’t see any non-garbage-collected candidates for the job.

This makes me a little bit uncomfortable, because, in general, I like to have options. It would be nice if, 10 years from now, I had more than one choice for heap management (presuming that we still have "heaps" in a decade…).

Although I like the concept of making garbage collection optional, it’s much harder to put that option into practice successfully. The trouble is that most people don’t just use programming languages, they also use the standard libraries and toolkits which support those languages. And while it’s fairly trivial to make use of garbage collection within a programming language optional, it’s much harder to do it well with a library. So I don’t think some kind of compiler switch or memory manager plugin will resolve this issue of choice; a real choice would be an environment constructed to work with manual memory release, and I don’t see anyone building that.

Share This | Email this page to a friend

There are many reasons why this is the case. Garbage collection is a core feature of functional programming, because functional programming features do not work well with the semantics of explicit memory release. It is telling that language implementors have found it easier and more correct to create real-time garbage collectors than to create and work with non-garbage-collected variants of functional languages.

I was unable to find a single example of a truly functional language which did not use garbage collection in any standard version. Haskell, OCaml, Common Lisp, Erlang, ML, Scheme, etc., are all garbage collected. Moreover, functional features are commonly added to languages like C#, Ruby, Python, ECMAScript, and others, which weren’t designed as functional languages, but happen to be garbage collected. Whereas languages like Delphi/Win32 get fewer such features, despite their continuing evolution. The FC++ library, which adds some functional programming features to C++, uses a reference-counting method of garbage collection internally, to its great benefit. The authors note:

For instance, compared to Läufer’s approach, we achieve an equally safe but more efficient implementation of the basic framework for higher order functions. This is done by allowing function objects to be multiply referenced (aliased), albeit only through garbage collected “pointers”. The difference in performance is substantial: compared to Läufer’s framework, and running with the same client code (the main example implemented by Läufer) we achieve a 4- to 8-fold speedup.

The D programming language is designed to combine the power of C++ with the productivity and contemporary features of Ruby Python. It supports function literals, closures, and lazy evaluation. It doesn’t use a runtime, or a JIT compiler, but it uses garbage collection for heap management.

Share This | Email this page to a friend

Let’s perform a thought experiment and see what would happen if this feature were added.

Win32 Interfaces Today

First, let’s examine in a bit more detail how interfaces work in Delphi for Win32 today, and the features for non-reference-counted interfaces which are already available.

All interfaces in Delphi in inherit from IInterface/IUnknown. This interface defines three methods, of which two are interesting in the context of reference counting, _AddRef and _Release. When an interface reference comes in scope, the compiler inserts a call to _AddRef, and when the reference goes out of scope, the compiler inserts a call to _Release. Both methods are called virtually, which means you need an intact VMT; in order for the call to succeed. This means that the instance on which they are called must either still be allocated, or a least that the memory holding their VMT has not been overwritten by other data.

You can define a class which supports interfaces and use those interfaces in a roughly non-reference-counted manner today, with certain limitations. If you implement _AddRef and _Release to both return -1, this is a convention (see comments in Classes.pas) to indicate that you’re ignoring reference counting, and you can implement _Release to not free the instance when the ref count goes to 0. Importantly, however, _AddRef and _Release are still going to be called for any non-nil interface reference, whether or not you’re managing lifetimes via reference counting. The functions don’t have to do anything, but they will be called anyway.

As I mentioned in my last post, it is a very common case to have multiple references of different interface types to a single instance of an object that supports those interfaces. This is because of the intentionally limited window into the object’s features that an interface provides. If you choose to "turn off" reference counting by making _AddRef and _Release return -1, then you must be very careful to manually nil all interface references to an instance before Free-ing the instance with an object reference. Should you forget to do that, or do it incorrectly, then you will very likely have to debug an access violation in System._IntfClear. This is usually difficult to do since the reference which raised the access violation is certainly out of scope, and is of type IInterface, which doesn’t provide a lot of information, especially as a dangling pointer. Despite these obstacles, I use non-reference-counted interfaces quite a bit, because interfaces are a useful programming technique, and I don’t want to live without them in cases where reference counting cannot be used, as with visual components, which are mostly lifetime-managed via the Ownership mechanism.

"Non-Reference-Counted" Win32 Interfaces

So even though we can in fact have an interface reference to something which is not lifetime managed via reference counting, this _AddRef and _Release thing is still a bit of a pain. Let’s imagine what would have happened if Chuck had implemented the suggestion to make IInterface not do any kind of reference count calls, and to introduce those in IUnknown, which would then be a subtype of IInterface, rather than an alias. IInterface would only define the QueryInterface method.

Now, we might think, that the compiler can insert calls to _AddRef and _Release when it sees an interface reference which is of the type which is a subtype of IUnknown, and not to insert those calls when it sees an interface reference which is of a type which is not a subtype of IUnknown. This seems like a pretty good idea, since it will certainly solve the access violation problem.

However, we quickly hit a roadblock. What if a class implements both "types" of interface?

With the present system, where _AddRef and _Release are always called, the compiler always knows what to do when it sees an interface reference. With our proposed new system, it’s less clear what the compiler should do. None of the options I can think of looks particularly good:

• The compiler could insert calls to _AddRef and _Release for a reference of an interface type which is a subtype of IUnknown, and not insert these calls for a reference of an interface type which is not a subtype of IUnknown. This is dangerous, for reasons which should be obvious. Far from reducing the confusion over different types of memory management, we’ve made it worse with this option.
• When the compiler encounters an interface reference of a type which is not a subtype of IUnknown, the compiler could call QueryInterface to see if the implementor supports IUnknown, and call _AddRef and _Release if it does. This means, among other things, that there is now very little advantage over the present method of implementing "non-reference-counted interfaces." In particular, the access violation problem is back. Worse, it means that the behavior of your application at the site of the interface reference can change when you implement a new interface on a class.
• The compiler could simply forbid any class from implementing both "types" of interface. This is probably the safest option, but it’s incredibly limiting and it violates the spirit and intention of an interface. It’s limiting because it forbids things which are already implemented in the VCL today, like TComponent’s interface support, which is designed to work both with reference counted and non-reference-counted instances. It violates the intention of an interface because it introduces implementation details into the signature of the interface. Interfaces are supposed to be interfaces only, not implementation. Having distinct "types" of interfaces leads to the same problems one encounters with multiple implementation inheritance, notably the "diamond problem." One could very well end up requiring parallel hierarchies of reference counted and non-reference-counted interface types, precisely the problem that interfaces are supposed to overcome.

On top of all of this, we still have the problems I referred to in my previous post, such as more dangling pointers.

The bottom line is that, far from solving the problem of conflicting systems of memory management, formal language support for a "non-reference-counted" root interface type in Win32 actually seems to make some of the worse. My conclusion is that the present state of the Delphi language is a better option than this proposed change.

Update: Clarified a couple of details regarding non-reference-counted interface features available today.

Share This | Email this page to a friend

For me, be general theme of most of the advantages of garbage collection can be summarized as "expressive programming." This means that your source code can be written to express the concise nature of the problem instead of precisely how to solve it. Most programmers do not write applications to solve the problem of memory management. Memory management, instead, is a kind of tax that we have to pay to get to our solution. Eliminating explicit memory management can allow the programmer to focus most directly on the problem at hand. As a loose analogy, consider the difference between a LINQ query and a traditional for loop. The former asserts the need to iterate, but delegates the method of iteration to the instance to be iterated. The traditional for loop, on the other hand, spells out the precise method of iteration. The LINQ query, therefore, is more expressive, since it does not pollute the description of what needs to be done with an algorithm for how it is to be done. The difference may seem kind of academic, until you examine how easy it is to parallelize the more expressive code. Then you see the advantage of expressive coating.

Expressing this point another way, manual memory management tends to force programmers to write their code in a way which suits the memory management as well is the problem instead of a way which most directly suits the problem. The most dramatic example of this is functional programming, a powerful technique which is extremely difficult to do well without garbage collection.

Garbage collection is often contrasted with "manual" memory management, as though any application which doesn’t use garbage collection frees all allocations with an explicit call to Free. But that isn’t true at all. In fact, Delphi has several different methods of memory management, depending upon what you’re doing. Off the top of my head:

• You can free an allocation with an explicit call to Free.
• Components can be freed via the Ownership mechanism. This is critical to streaming.
• Components will be freed when their Parent is freed.
• Some types are allocated on the stack, and freed when they go out of scope.
• Some types are reference counted.

Now, that’s a lot of different methods of handling memory, and I’m reasonably sure that I have not had all of them. Worse, they can and do conflict. This is most commonly noticed when using interfaces, and though it’s hardly the only example, it is perhaps the one with the greatest potential to cause problems. So much so, in fact, that I believe it is principally responsible for the fact that interfaces are not widely used in Win32 Delphi code (in contrast, for example, with .NET code), even though they were introduced in Delphi 3.

For just one example of how this conflict between memory management types complicates Win32 programming in Delphi, look closely at the constructor for TXMLDocument. Another is the lengths that you must take to ensure that you explicitly nil all interface references to a non–reference-counted instance before Freeing it, or you will almost certainly see an Access Violation in _IntfClear.

People occasionally suggest that reference counting is a holdover from COM, and a solution to this problem would be a non–reference-counted interface type. It’s not a bad idea in its own right, but it doesn’t really solve the problem of memory management. For various reasons, interfaces and reference counting are useful for any cross language interaction, not just COM. But even without cross language interaction, some kind of automatic memory management is still useful when using interfaces. The nature of how interfaces work is well-suited to garbage collection. Given an interface reference, you’re not really supposed to do low-level operations on the instance. Moreover, because of the limited window into the instance but an interface provides, it is the rule rather than the exception, when working with interfaces, but an application will have multiple references to an instance. If a particular class implements IFoo and IBar, you will very likely have one reference of each type at some point in the execution of the program. This is in contrast to object references, where it is more common to have a single reference to an instance, since the type of that reference probably does everything you need.

Contrast the five different types of memory management I’ve listed with a much simpler scenario for Delphi for .NET. There, you have only garbage collection of the heap and stack-based allocations, and they never conflict. Your code focuses on the task at hand, rather than when and how to release each instance you create.

Since a few people seem to read any assertion that garbage collection could possibly have any single redeeming value as a threat to their programming worldview, I will restate that this post is not an assertion that every programmer should give up manual memory management for all tasks, forever. Rather, it’s a description of a single feature that I like.

Share This | Email this page to a friend

Places where GUIDs are useful include:

• Surrogate key values
• Win32 Delphi interfaces
• COM registration

Some places where GUIDs are  completely, utterly unfit include:

• Directory names
• URLs
• Product registration codes

Share This | Email this page to a friend