Category Archives: Best Practice

Testing Spring Controllers with Spring Security, Embedded LDAP, and Different Application Contexts

To test a secured Spring application with Spring Security and LDAP authentication, we need to take the following steps:

  1. Activate and configure your embedded LDAP server for your tests.
  2. Define users and roles of your choice for your embedded LDAP server.
  3. Write Spring Tests.

Activate and configure your embedded LDAP server

There are at least the two following possibilities to start an embedded LDAP server with Spring:

  1. With Spring Boot
  2. With Spring Security

With Spring Boot

Spring Boot starts for each application context an embedded LDAP server automatically if Spring Boot finds an LDAP server implementation in the classpath and you declare some configuration properties in the application.properties. For the LDAP server implementation called unboundid, you need to declare at least the following properties:

spring.ldap.embedded.ldif=classpath:bootstrap.ldif
spring.ldap.embedded.base-dn=dc=springframework,dc=org

# Further properties are:
spring.ldap.embedded.port
spring.ldap.embedded.validation.enabled
spring.ldap.embedded.credential.username
spring.ldap.embedded.credential.password
spring.ldap.embedded.ur

As highlighted above, Spring Boot starts an embedded LDAP server for each application context. Logically, that means, it starts an embedded LDAP server for each test class. Practically, this is not always true since Spring Boot caches and reuses application contexts. However, you should always expect that there is more than one LDAP server running while executing your tests. For this reason, you may not declare a port for your LDAP server. In this way, it will automatically uses a free port. Otherwise, your tests will fail with “Address already in use”.

With Spring Security

If you want to use – for your tests – an embedded LDAP server as your central authentication management system with Spring Security, then you need to configure Spring Security as follows. Tip: Use two Spring Security configuration classes: one in your src/main/java and one in your src/test/java. The one in src/main/java could simply connect to an existing, non-embedded LDAP server. The one in src/test/java starts one embedded LDAP server for each new test context.

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.config.http.SessionCreationPolicy;
import org.springframework.security.crypto.factory.PasswordEncoderFactories;
import org.springframework.security.crypto.password.PasswordEncoder;

@Configuration
@EnableWebSecurity
public class SecurityTestConfiguration extends WebSecurityConfigurerAdapter {

	private static final Logger LOGGER = LoggerFactory.getLogger(SecurityTestConfiguration.class);

	@Override
	protected void configure(AuthenticationManagerBuilder auth) throws Exception {
		PasswordEncoder passwordEncoder = PasswordEncoderFactories.createDelegatingPasswordEncoder();
		// @formatter:off
		auth.ldapAuthentication()
			.userDnPatterns("uid={0},ou=people")
			.groupSearchBase("ou=groups")
			.contextSource()	// starts an LDAP server if url is not provided
				.ldif("classpath:bootstrap-spring-security.ldif")
				.root("dc=springframework,dc=org")
				// automatically adds the entry indicated by .root()
				.and()
			.passwordCompare()
				.passwordEncoder(passwordEncoder)
				.passwordAttribute("userPassword")
			.and();
		// @formatter:on
		LOGGER.info("Security configuration loaded.");
	}

	@Override
	protected void configure(HttpSecurity http) throws Exception {
		// @formatter:off
		http.sessionManagement()
			.sessionCreationPolicy(SessionCreationPolicy.STATELESS)
			.and();
		http.authorizeRequests()
			.anyRequest().authenticated()
			.and()
		.httpBasic();
		// @formatter:on
	}
}

If you do not declare a URL, but an LDIF file path instead, then Spring Security starts an embedded LDAP server automatically – again – for each application context.

One very important difference to Spring Boot is that the root entry is automatically created by Spring Security! So, your LDIF file must not contain the root entry. Otherwise, your application or your tests will fail with “An entry with DN ‘<dn of your root entry>’ already exists in the server.” We highlight this critical root entry in the next section.

Define users and roles of your choice for your embedded LDAP server

Below, you find an example LDIF file. The first entry is the root entry whose dn has to be used as value for the property spring.ldap.embedded.base-dn or, respectively, as argument for the method root(). Note that Spring Security automatically adds the root entry to the LDAP server. Hence, your LDIF file may not contain the root entry.

dn: dc=springframework,dc=org
objectclass: top
objectclass: domain
objectclass: extensibleObject
dc: springframework

dn: ou=groups,dc=springframework,dc=org
objectclass: top
objectclass: organizationalUnit
ou: groups

dn: ou=subgroups,ou=groups,dc=springframework,dc=org
objectclass: top
objectclass: organizationalUnit
ou: subgroups

dn: ou=people,dc=springframework,dc=org
objectclass: top
objectclass: organizationalUnit
ou: people

dn: ou=space cadets,dc=springframework,dc=org
objectclass: top
objectclass: organizationalUnit
ou: space cadets

dn: ou=\"quoted people\",dc=springframework,dc=org
objectclass: top
objectclass: organizationalUnit
ou: "quoted people"

dn: ou=otherpeople,dc=springframework,dc=org
objectclass: top
objectclass: organizationalUnit
ou: otherpeople

dn: uid=ben,ou=people,dc=springframework,dc=org
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
cn: Ben Alex
sn: Alex
uid: ben
userPassword: {SHA}nFCebWjxfaLbHHG1Qk5UU4trbvQ=

dn: uid=bob,ou=people,dc=springframework,dc=org
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
cn: Bob Hamilton
sn: Hamilton
uid: bob
userPassword: {noop}bobspassword

dn: uid=joe,ou=otherpeople,dc=springframework,dc=org
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
cn: Joe Smeth
sn: Smeth
uid: joe
userPassword: joespassword

dn: cn=mouse\, jerry,ou=people,dc=springframework,dc=org
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
cn: Mouse, Jerry
sn: Mouse
uid: jerry
userPassword: jerryspassword

dn: cn=slash/guy,ou=people,dc=springframework,dc=org
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
cn: slash/guy
sn: Slash
uid: slashguy
userPassword: slashguyspassword

dn: cn=quote\"guy,ou=\"quoted people\",dc=springframework,dc=org
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
cn: quote\"guy
sn: Quote
uid: quoteguy
userPassword: quoteguyspassword

dn: uid=space cadet,ou=space cadets,dc=springframework,dc=org
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
cn: Space Cadet
sn: Cadet
uid: space cadet
userPassword: spacecadetspassword

dn: cn=developers,ou=groups,dc=springframework,dc=org
objectclass: top
objectclass: groupOfUniqueNames
cn: developers
ou: developer
uniqueMember: uid=ben,ou=people,dc=springframework,dc=org
uniqueMember: uid=bob,ou=people,dc=springframework,dc=org

dn: cn=managers,ou=groups,dc=springframework,dc=org
objectclass: top
objectclass: groupOfUniqueNames
cn: managers
ou: manager
uniqueMember: uid=ben,ou=people,dc=springframework,dc=org
uniqueMember: cn=mouse\, jerry,ou=people,dc=springframework,dc=org

dn: cn=submanagers,ou=subgroups,ou=groups,dc=springframework,dc=org
objectclass: top
objectclass: groupOfUniqueNames
cn: submanagers
ou: submanager
uniqueMember: uid=ben,ou=people,dc=springframework,dc=org

Write Spring Tests

And here are two example tests which tests the same thing but each with a different application context (see the different class annotations). As you will notice in the console output, two LDAP servers with different ports will be started and the tests will pass.

import static org.hamcrest.Matchers.*;
import static org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;

@RunWith(SpringRunner.class)
@SpringBootTest
@AutoConfigureMockMvc
public class HelloController1Test {

	@Autowired
	private MockMvc mvc;

	@Test
	public void getHelloWithValidLogin() throws Exception {
		// @formatter:off
		mvc.perform(get("/").with(httpBasic("bob", "bobspassword")).with(csrf()))
			.andExpect(status().isOk())
			.andExpect(content().string(equalTo("Greetings from Spring Boot!")));
		// @formatter:on
	}

	@Test
	public void getHelloWithInvalidPassword() throws Exception {
		MockHttpServletRequestBuilder requestBuilder = MockMvcRequestBuilders.get("/")
				.accept(MediaType.APPLICATION_JSON);

		mvc.perform(requestBuilder.with(httpBasic("bob", "wrong password"))).andExpect(status().isUnauthorized());
	}
}
import static org.hamcrest.Matchers.*;
import static org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.data.ldap.AutoConfigureDataLdap;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.context.annotation.Import;
import org.springframework.http.MediaType;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;

import chw.tutorial.springboot.SecurityTestConfiguration;

@RunWith(SpringRunner.class)
@WebMvcTest(value = HelloController.class)
@Import(SecurityTestConfiguration.class)
@AutoConfigureDataLdap
public class HelloController3Test {

	@Autowired
	private MockMvc mvc;

	@Test
	public void getHelloWithValidLogin() throws Exception {
		// @formatter:off
		mvc.perform(get("/").with(httpBasic("bob", "bobspassword")).with(csrf()))
			.andExpect(status().isOk())
			.andExpect(content().string(equalTo("Greetings from Spring Boot!")));
		// @formatter:on
	}

	@Test
	public void getHelloWithInvalidPassword() throws Exception {
		MockHttpServletRequestBuilder requestBuilder = MockMvcRequestBuilders.get("/")
				.accept(MediaType.APPLICATION_JSON);

		mvc.perform(requestBuilder.with(httpBasic("bob", "wrong password"))).andExpect(status().isUnauthorized());
	}
}

Pitfalls

As described above, you may get one of the following error messages if you have not correctly configured your embedded LDAP server. In this blog post, we explained why these errors occur and gave appropriate solutions.

  1. “Address already in use: NET_bind”
  2. “com.unboundid.ldap.sdk.LDAPException: An entry with DN ‘dc=springframework,dc=org’ already exists in the server.”

Pitfalls with “Convention over Configuration”

The paradigm Convention over Configuration is a great way to develop software (web) application. I like it…IFF it is implemented well.

Problem

Unfortunately, many frameworks which rely on this paradigm are hard to work with. One example is Spring Boot. Users who are new to Spring Boot must read many pages of documentation to learn the “sensible defaults”. These defaults are often not intutive to new users because the users do not yet know the underlying concepts for which the defaults are ment to be. Hence, the first steps with Spring Boot take time – more time than necessary.

Moreover, the defaults seem to work in a magical way – just by putting some new annotations at a class or a method. The javadoc often does not explain how the annotations work. It’s so easy to add a short text that describes that Spring searches the classpath for annotations.

Furthermore, annotations can also cause incompatibility issues with other annotations and conventions. For example, the annotation @DataJpaTest disables full auto-configuration such that you wonder why auto-wiring is not working anymore. Although the javadoc mentions this issue, it is unnecessarily complicated. I wish that adding annotations only adds behavior.

Finally, you can often do one thing in more than one way. Usually, basic annotations are provided which are used by aggregated annotations to cover recurring use cases. This approach often results in annotated classes which have several basic annotations twice or thrice. Although this approach does not influence the functionality, it is really confusing for (new) users who need to write some piece of code: when should I use which (aggregated) annotations? Users who read the code could ask themselves: why did the author of this code add these annotations more than once?

Proposed Solution

  1. Uncover all conventions, e.g., by a configuration file with default values, such that the conventions are visible for (new) users. In this way, the user can read through the config file and learn what features and conventions are provided by the framework. We follow this approach with our framework Kieker.
  2. Each annotation should explain how it is processed. If you have a bunch of similar annotations, either copy and paste the same text or add a javadoc link to the processing engine where the approach is described once at a central place.
  3. Let annotations only add new behavior. Let them not disable other features.
  4. Only provide one way to do something to reduce confusion and time to read/understand. When the (new) user has several possibilities to do the same thing, it is laborious to understand when to use which approach. If there is no other choice, provide at least recommendations when to use which approach.

Progress Monitoring of Loops

When coding a loop, we often insert some logging code for debugging or progress monitoring purposes. The following code snippet shows an example:

for (int i = 0; i < invocations.size(); i++) {
  // loop body begin
  ...
  // loop body end
  System.out.println("current index: "+i); // pure java
  currentTimeInMs = ...;
  if (currentTimeInMs % 1000 == 0) {  // log every econd
    logger.debug("current index: {}", i);    // slf4j
  }
}

Disadvantages of the Naive Approach

However, this approach has some disadvantages. First, the logging code is executed by the same thread which executes the loop body. If the logging code causes an exception, for example, due to a NullPointerException, the execution of the loop body is interrupted. Thus, the execution of the application can break because of some fault in the logging code.

Moreover, the executing thread is thwarted by the addtional logging code. Remember that executing I/O operations, like printing to the console or to a file, is slower by several orders of magnitude. Especially for performance critical code regions, this approach is not recommended.

Finally, the logging code is executed in each and every iteration of the loop although it is often sufficient to log the loop’s state every second or every minute. If you now think of a solution similar to the one shown above in Line 5, then remember that this additional if statement causes even more runtime overhead and further increases the potential for errors.

A Better Approach

A less intrusive, less error-prone, and faster approach is the following. First, we declare a volatile field progressIndex which represents the current index of the loop we want to monitor. For this purpose, we update the value of this field as shown in the following code snippet. We do not insert our original logging code here.

for (int i = 0; i < invocations.size(); i++) {
    progressIndex = i;
    // loop body
}

Instead, we create an additional thread which executes our logging code now. This thread performs a do-while loop as shown in the following code snippet from Line 7 to Line 28. We use `Thread.sleep()` to implement a user-defined time interval (here: 1000 ms). The thread reads the current index of the loop under monitoring indicated by the variable progressIndex in Line 13. As we declared it as volatile, it is synchronized between our new monitoring thread and the application thread(s). We cache the progressIndex as local variable, since the value of this variable can change on each subsequent read-access. In this way, the value is consistent within an iteration of the do-while loop.

To avoid interfering with the application, we declare our new monitoring thread as a deamon. In this way, the thread is automatically terminated by the JVM when all application threads have terminated.

The rest of theĀ do-while loop computes the remaining execution time of the application and executes the domain-specific logging code. In this example, we print out the current number of NoSql queries which were already executed by the application loop.

Thread progressMonitoringThread = new Thread(new Runnable() {
    @Override
    public void run() {
        long lastTimestamp = System.currentTimeMillis();
        int lastProgressIndex = progressIndex;
        
        do {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                throw new IllegalStateException(e);
            }
            final int localProgressIndex = progressIndex;
            
            long currentTimestamp = System.currentTimeMillis();
            long durationInMs = currentTimestamp - lastTimestamp;
            int count = localProgressIndex - lastProgressIndex;
            if (count <= 0) continue;
            long durationInMsPerElement = durationInMs / count;
            
            long remainingDurationInMs = durationInMsPerElement * (numQueries - (localProgressIndex + 1));
            String message = String.format("Executing query %s/%s (%s sec. remaining)", localProgressIndex + 1,
            numQueries, Duration.ofMillis(remainingDurationInMs).getSeconds());
            System.out.println(message);
            
            lastTimestamp = currentTimestamp;
            lastProgressIndex = localProgressIndex;
        } while (progressIndex < numQueries);
    }
});
progressMonitoringThread.setDaemon(true);

progressMonitoringThread.start()

This approach does not break the execution of the application if the logging code is faulty. In the worst case, our monitoring thread terminates – no interfering with the application. Moreover, the runtime overhead is reduced to a minimum. We only set the progressIndex in each iteration of the application loop. Finally, the approach allows to log the state of the loop at a user-defined time interval. We are not forced to log it on each iteration anymore. Hence, we have changed the type of notification from an event-driven style to a time-driven style.