code

photography of three dogs looking up

Optimizing Away a LEFT JOIN in a Pinch

A couple years ago the company I worked for had had just launched a free consumer-facing application, and had been pushing it hard on social media. They were seeing hundreds to thousands of active users. It was painfully slow, and the culprit appeared to be the database. One of the main queries on the application was pushing the database server to its limits. Some users […]

Optimizing Away a LEFT JOIN in a Pinch Continue Reading »

Quickly Testing A Refactored SQL Query for Equivalence

A common problem I have worked on in my career has been to troubleshoot and improve slow, complex SQL queries. Usually these are part of codebases with few or no tests. If there are tests, they usually are testing some other part of the code and assume that the query is correct. It is rare to see tests that prove the columns being selected are

Quickly Testing A Refactored SQL Query for Equivalence Continue Reading »

Condensing Tests in Kotlin : Dependencies

My current team maintains an application built using Spring and Kotlin. We have several thousand unit and integration tests. I’m a big fan of condensing code as much as possible, so long as it remains legibile. A lot of our older integration tests in particular take the following form: Using default Kotlin formatting, the dependencies for the tests take up 3 lines each. This could

Condensing Tests in Kotlin : Dependencies Continue Reading »

Database Schema Migrations : A Few Lessons Learned

About 5 years ago I was introduced to Flyway, a tool for managing schema migrations using simple SQL scripts. I’ve used it on several projects now, and have more recently been introduced to Liquibase. The two tools both solve similar problems, with slightly different approaches. The main purpose of these tools is to version changes to the schema of the databasse. Liquibase has a good

Database Schema Migrations : A Few Lessons Learned Continue Reading »

The One Line Docker Change for UTF-8

At work we have been trying to ensure that all of our applications correctly support UTF-8. This includes making sure that our REST APIs can handle accented characters (ex: é) and emoji (?) when it makes sense to. This is somewhat complicated by the multiple database platforms we have (MSSQL, MySQL, and Postgres). MSSQL is fairly straight forward – it usually involves swapping the text

The One Line Docker Change for UTF-8 Continue Reading »

Troubleshooting a Performance Problem Using NewRelic

Or, how to poorly implement adding css and javascript resources to a page One of the tools we use at work is the MagnoliaCMS. It powers some portions of some of our properties. For those who have not heard of it (I had not until a few years ago), think of it as WordPress, but more enterprise-y. Recently we noticed that some of the properties

Troubleshooting a Performance Problem Using NewRelic Continue Reading »

Switching from Essentials4J to Kotlin

At work we had a project that used Essentials4J and its predecessor Rapidoid Fluent, to simplify some stream/collection APIs. We were converting this project to Kotlin, and no longer need that library due to features of Kotlin’s standard library. The conversion was actually fairly straightforward. Do.group(…).by(…) Do.map(…).to(…) Do.map(…).to(…) with unique values d639a7ccf9252433

Switching from Essentials4J to Kotlin Continue Reading »

Getting a quick hash of a query in PostgreSQL

I’ve been doing a lot of tweaking of SQL queries as of late, trying to squeeze some performance out of a query by adding indexes, optimizing joins, and some other operations. To do a quick check to ensure that the queries return the same results, I’ve been hashing the results to see if I get an identical hash back. This solution was adapted from https://stackoverflow.com/a/13948327/206480.

Getting a quick hash of a query in PostgreSQL Continue Reading »

PostgreSQL’s GREATEST (and LEAST) function

PostgreSQL has two functions that are not standard SQL, but are very helpful in certain situations : GREATEST and LEAST. In SQL MIN and MAX are aggregate functions that operate across all rows. But there are times where you want the smallest or largest value in a finite number of columns, a scalar min/max. That is where GREATEST and LEAST come in. Both allow for

PostgreSQL’s GREATEST (and LEAST) function Continue Reading »

Automatic file encoding detection in Java

A few months ago I worked on a process that imports Facebook Leads into a legacy system. Facebook sends its advertising data as UTF-16 encoded CSV. The tool also had to support the CSV files occasionally being ended by hand, which reverted the encoding to something a bit more standard. Thankfully, there was a small library out there that helped. So, in case you ever

Automatic file encoding detection in Java Continue Reading »

Salesforce Trigger Context

At work we are working on a new Salesforce organization. We’ve written some code over the last few months, but at the advice of some contractors who are more seasoned with Apex, we are doing some refactoring of the code. One of these changes is making sure that recursive triggers are not possible – something along the lines of Account AFTER INSERT being called, which

Salesforce Trigger Context Continue Reading »

Coldfusion 9 ORM, Caching and Autocommit

One more from the archives of the company dev blog, this time from August 2011. We have been gradually moving off of ColdFusion over the last several years, but maybe there is something in here that might be useful for someone. We have been using ColdFusion 9 for a few months now. With all new code that is developed, we have been abandoning <CFQUERY> in

Coldfusion 9 ORM, Caching and Autocommit Continue Reading »

Performance of CFScript

This is another article from the retired company blog. This one is from September, 2011. I have been developing ColdFusion on and off for about 10 years, and one topic that has been controversial the entire time is CFScript. The arguments go back and forth: ‘It’s too slow’, ‘it’s not ColdFusion’, ‘It looks like Javascript, but isn’t’. On our team, the majority of our code

Performance of CFScript Continue Reading »

Logging in To Salesforce from ColdFusion

Im continuing to clear out articles from the retired company development team blog. This one, was instrumental in getting me connected with one of my first big side work projects, which evolved into a two year project that helped us retire our mortgage. From August 2011: We have been working the last few months on an experiment with Salesforce.com. The code is some of our

Logging in To Salesforce from ColdFusion Continue Reading »

Limiting the Use of CAPTCHA

I used to blog periodically on my company’s blog. Earlier this year, the blog was taken down, and I needed to reference something I posted from 2012. I’ve restored the post using the Wayback machine here. The information is a little dated, but maybe it could be useful to someone. This post references the old reCAPTCHA system that Google used to verify book scans and

Limiting the Use of CAPTCHA Continue Reading »

Optimizing a SQLite Database

TL:DR; takeaways: These seem very obvious, but with rushed deadlines, and “it works well enough”, these things are sometimes overlooked. The story I’ve been working on a project that involves taking flat files, importing them into SQLite, and exporting that database for use on low-powered client devices. I’ve had two such projects over the past couple of years, and each one has provided some small

Optimizing a SQLite Database Continue Reading »

Using Consumer in Spring Validator to validate nested collections

In one of my projects I have a custom Spring Validator that validates a nested object structure, and adds per-field error messages. As an example, a field nested inside an array might produce an error like the following: The Errors object works as a stack, so field names have to be pushed as the validator iterates through arrays and nested objects. So, a very simple validator

Using Consumer in Spring Validator to validate nested collections Continue Reading »

Hibernate Logging Options

For years I have relied on a simple property “show-sql=true” to see Hibernate’s generated SQL. This option is fairly limited: it bypasses the logging framework in the rest of my apps (SLF4J) it doesnt show parameters, just question marks it doesn’t show any timing information Frustrated at these limitations, I set out to understand all of the other options available to me. This is very

Hibernate Logging Options Continue Reading »

Use SQL to Find Central ZIP Codes for Metro Areas

At work we have a copy of the zip-codes.com Business database, which I tend to reference somewhat frequently due to the nature of projects I work on. Today I needed a list of postal codes from major metro areas which would be used to drive a test data generator. The data generator has access to a simpler, non-commercial zip code list, and can do radius

Use SQL to Find Central ZIP Codes for Metro Areas Continue Reading »

Salesforce: Serializing Objects from a Managed Package

I’m currently working on a Salesforce project that utilizes a few home-built managed packages. There are lots of restrictions of what you can and can’t do with managed package, and most of those are documented fairly clearly. Some, however, are somewhat hidden. One such restriction is this: Only custom objects, which are sObject types, of managed packages can be serialized from code that is external

Salesforce: Serializing Objects from a Managed Package Continue Reading »

Naming a class in Salesforce the same as a built in interface

I am currently working on a small managed package for a Salesforce project. One of the nice things you can do inside of managed packages is create post-install scripts, which are sort of like migrations in Rails. For some reason, I kept getting a compiler error when I created even the simples of install handlers: global class PostInstallHandler implements InstallHandler {
 global void onInstall(InstallContext context)

Naming a class in Salesforce the same as a built in interface Continue Reading »

Troubleshooting a SQL Server Implicit Conversion Issue

Over the last couple of weeks, I have been spending some time troubleshooting performance issues in some of our Java-based RESTful services. This week I came across one that required some different steps to troubleshoot. First, the tl;dr – our JDBC driver (jTDS) converts CHAR to NCHAR, which causes an index SCAN on a CHAR column instead of an index SEEK. The Architecture Here is

Troubleshooting a SQL Server Implicit Conversion Issue Continue Reading »

Java 8: Optional

A fellow developer today asked me a question about the Optional interface in Java 8. My team is still working on a Java 6 stack, but his team is blazing the trail to Java 8. I’ve used Optional a little bit in some side work, and I am a little more familiar with Guava’s version. His use case centered around the correct syntax to rewrite this using map() instead

Java 8: Optional Continue Reading »

Mockito AdditionalAnswers

Sometimes you use a framework for years, and then discover something new that it can do. I had one of those moments today with Mockito. I have an API that I am mocking, where I need to capture the argument passed in for further testing, that had a line that ended up looking something like this: This is actually incorrect Mockito syntax. The captor.capture() doesn’t go

Mockito AdditionalAnswers Continue Reading »

IE9 Print Stylesheet Issues

Over the past two years, I have had the pleasure of fixing two issues with printing from Internet Explorer 9. Both of the problems deal with IE-specific “filter” attributes in CSS. We have an application that is used by an audience that deals with a lot of paperwork. They like to print information from our application to add to their paper files, so it is important that printing functionality works

IE9 Print Stylesheet Issues Continue Reading »

black and yellow analog clock

Mocking Time with Joda

A big project I have been developing over the last year has been an automated billing system. There is a part of it that involves scheduling bills, and has a lot of pieces that need to know various dates – when bills are due, when they were created, and the current time. We are trying to write good unit tests, and I am trying to make sure the

Mocking Time with Joda Continue Reading »

Encrypting config files with Spring 3.1 and jasypt

I’ve been working on a project that at one point needed encrypted configuration files. The JASYPT library provides a very nice tie-in with Spring. There were some pieces I had to put together, so here are the results of my @Configuration file that handles encryption. It will scan a given directory (specified by the VM argument config.dir) for any .properties files, and uses JASYPT to

Encrypting config files with Spring 3.1 and jasypt Continue Reading »

Full Spring 3.1 Config

Someone asked for my full Spring 3.1 annotation configuration. I’ve stripped all domain-specific information, but the overall structure is intact. SpringConfig.java – this is the top level class, is empty except for @ComponentScan and @Import statements. The web.xml references this. SpringMvcConfiguration – Any MVC related configuration DatabaseConfiguration @Configuration
@ComponentScan(basePackageClasses = { SpringConfig.class})
@Import({ SpringMvcConfiguration.class, DatabaseConfiguration.class})
public class SpringConfig {

}
 @Configuration
@EnableWebMvc
@Import({ MvcComponents.class,BeanConfiguration.class })
public class SpringMvcConfiguration extends WebMvcConfigurerAdapter {
 @Qualifier(“generalMapper”)


Full Spring 3.1 Config Continue Reading »

Spring 3.1, No-XML, Hibernate, Cglib, and PermGen errors

Lately I have been maintaining several Spring-MVC applications written from the ground up with Spring 3.1. They use the purely Java based configuration scheme that comes in version 3.1, Hibernate. The apps do not have the pattern of “an interface for every class” that some Spring apps have, so it proxies concrete classes using Cglib. When the apps are deployed to Tomcat, we do a

Spring 3.1, No-XML, Hibernate, Cglib, and PermGen errors Continue Reading »

Making Java, Coldfusion, Tomcat and PayflowPro Play Nicely

One of the more odd parts of our architecture at work involves a cluster of Tomcat instances running ColdFusion and Java services side by side. We are porting our existing ColdFusion services over to Java/SpringMVC applications, and during the transition they are being served up by the same app servers. One of these services interacts with Paypal/PayflowPro. We have a ColdFusion Component (CFC) that makes

Making Java, Coldfusion, Tomcat and PayflowPro Play Nicely Continue Reading »