Thursday, November 30, 2006

Struts: Paging and Sorting with Displaytag

In the previous post, I described the use of Displaytag to implement paging in a simple JSP. In this example, I describe the use of Displaytag to implement sorting along with paging in Struts.
Skip to Sample Code
In this example, we take a single input field, which is used to filter the employee list based on the minimum salary. Follow these steps to implement the solution.
  1. Start by importing struts-blank.war file into Eclipse.
  2. Follow the configuration steps 1, 2, 4, 5, and 6 in from the "Pagination with Displaytag" post.
  3. Create the search page as shown below
    <%@ page language="java" contentType="text/html; charset=ISO-8859-1" pageEncoding="ISO-8859-1"%>
    <%@ taglib uri="http://struts.apache.org/tags-bean" prefix="bean"%>
    <%@ taglib uri="http://struts.apache.org/tags-html" prefix="html"%>
    <%@ taglib uri="http://struts.apache.org/tags-logic" prefix="logic"%>
    <%@ taglib uri="http://displaytag.sf.net" prefix="display"%>
    <%@ page import="beans.Employee,data.DAO,java.util.List,org.displaytag.tags.TableTagParameters,org.displaytag.util.ParamEncoder"%>
    <html:html>
    <head>
    <title>Search page</title>
    <link rel="stylesheet" type="text/css"
    href="/StrutsPaging/css/screen.css" />
    </head>
    <body bgcolor="white">
    <html:form action="/search.do">
    <table>
    <tr>
    <td>Minimum Salary:</td>
    <td><html:text property="minSalary"></html:text></td>
    </tr>
    <tr>
    <td colspan="2"><html:submit property="submit" /></td>
    </tr>
    </table>
    </html:form>
    <logic:notEqual name="empList" value="null">
    <jsp:scriptlet>
    if (session.getAttribute("empList") != null) {
    String sortBy = request.getParameter((new ParamEncoder("empTable")).encodeParameterName(TableTagParameters.PARAMETER_SORT));
    DAO.sort((List) session.getAttribute("empList"), sortBy);
    }
    </jsp:scriptlet>

    <display:table name="sessionScope.empList" pagesize="4" id="empTable" sort="external" defaultsort="1" defaultorder="ascending">
    <display:column property="empId" title="ID" sortable="true" sortName="empId" headerClass="sortable" />
    <display:column property="empName" title="Name" sortName="empName" sortable="true" headerClass="sortable" />
    <display:column property="empJob" title="Job" sortable="true" sortName="empJob" headerClass="sortable" />
    <display:column property="empSal" title="Salary" sortable="true" headerClass="sortable" sortName="empSal" />
    </display:table>
    </logic:notEqual>
    </body>
    </html:html>
    pages/search.jsp
    Note that
    1. The display:table tag has the sort attribute defined as "external".
    2. Since the sort type is external, we have to provide for the actual sorting, which I implemented in the DAO class itself (see below).
    3. The column to sort by is obtained by the following peice of code
      request.getParameter((new ParamEncoder("empTable")).encodeParameterName(TableTagParameters.PARAMETER_SORT))
  4. Create the Action class and Form bean as shown below.
    public class SearchAction extends Action {
    public ActionForward execute(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse httpservletresponse) throws Exception {
    if (form == null) {
    return mapping.findForward("success");
    }
    try {
    SearchForm searchForm = (SearchForm) form;
    if (searchForm.getMinSalary().equals("")) {
    return mapping.findForward("success");
    }
    long minSal = Long.parseLong(searchForm.getMinSalary());
    List data = DAO.getData(minSal);

    request.getSession().setAttribute("empList", data);

    } catch (Exception e) {
    e.printStackTrace();
    }
    return mapping.findForward("success");
    }
    }
    actions.SearchAction
    public class SearchForm extends ActionForm {
    private String minSalary;
    public String getMinSalary() {
    return minSalary;
    }
    public void setMinSalary(String minSalary) {
    this.minSalary = minSalary;
    }
    }
    forms.SearchForm.java
  5. Modify the struts-config.xml to include the action and actionform as shown below
    <form-beans>
    <form-bean name="searchForm" type="forms.SearchForm" />
    </form-beans>
    <action-mappings>
    <action path="/Welcome" forward="/pages/Welcome.jsp" />
    <action name="searchForm" path="/search"
    type="actions.SearchAction" scope="session">
    <forward name="success" path="/pages/search.jsp"></forward>
    </action>
    </action-mappings>
  6. Create the DAO class as shown below
    public class DAO {
    public static List getData(long minSal) {
    SessionFactory sessionFactory = new Configuration().configure().buildSessionFactory();
    Session session = sessionFactory.getCurrentSession();
    List result = null;
    try {
    session.beginTransaction();
    result = session.createQuery("from Employee as emp where emp.empSal >=?").setLong(0, minSal).list();
    System.out.println("Result size : " + result.size());
    session.getTransaction().commit();
    } catch (Exception e) {
    e.printStackTrace();
    }
    return result;
    }

    public static List sort(List<Employee> list, String sortBy) {
    Comparator comp = getComparator(sortBy);
    Collections.sort(list, comp);
    return list;
    }

    private static Comparator getComparator(String sortBy) {
    System.out.println("Sort by : " + sortBy);
    if (sortBy ==null) {
    return new NameComparator();
    }
    if (sortBy.equals("empName"))
    return new NameComparator();
    if (sortBy.equals("empId"))
    return new IdComparator();
    if (sortBy.equals("empSal"))
    return new SalComparator();
    if (sortBy.equals("empJob"))
    return new JobComparator();

    return null;

    }

    private static class NameComparator implements Comparator {
    public int compare(Object emp1, Object emp2) {
    Employee employee1 = (Employee) emp1;
    Employee employee2 = (Employee) emp2;
    return employee1.getEmpName().compareTo(employee2.getEmpName());
    }
    }

    private static class IdComparator implements Comparator {
    public int compare(Object emp1, Object emp2) {
    Employee employee1 = (Employee) emp1;
    Employee employee2 = (Employee) emp2;
    return new Long(employee1.getEmpId()).compareTo(new Long(employee2.getEmpId()));
    }
    }

    private static class SalComparator implements Comparator {
    public int compare(Object emp1, Object emp2) {
    Employee employee1 = (Employee) emp1;
    Employee employee2 = (Employee) emp2;
    return new Long(employee1.getEmpSal()).compareTo(new Long(employee2.getEmpSal()));
    }
    }

    private static class JobComparator implements Comparator {
    public int compare(Object emp1, Object emp2) {
    Employee employee1 = (Employee) emp1;
    Employee employee2 = (Employee) emp2;
    return employee1.getEmpJob().compareTo(employee2.getEmpJob());
    }
    }

    }
    DAO.java
    Note here that
    1. The list sorting is done in this class itself.
    2. The Comparators are used to compare order the list based on each individual field (there may be scope for improvement here).

Pagination with DisplayTag

Displaytag is an opensource tag library that can be used to display tables on JSPs. Apart from being able to display tables, the displaytag library also has support for JSR-168 compliant portals through the "Display portal compatibility library", and also supports exporting tables to Excel through the "Excel export module". The following example demonstrates the use of DisplayTag to display a long list as a multi-page table. For this example, I used the default EMP table from the sample database which will be built at during Oracle installation. This example uses Oracle 10g R2, Java 5, Tomcat 5.5 and Hibernate 3.2.
Skip to Sample Code
To run the example, follow these steps:
  1. Download Displaytags from here, and include the displaytag-1.1.jar file in your classpath.
  2. Download the latest version of hibernate from hibernate.org, and include all the required jars in your classpath.
  3. Create the pagingEmp.jsp page as shown below
    <jsp:root version="1.2" xmlns:jsp="http://java.sun.com/JSP/Page"
    xmlns:display="urn:jsptld:http://displaytag.sf.net">
    <jsp:directive.page contentType="text/html; charset=UTF-8" />
    <link rel="stylesheet" type="text/css" href="css/screen.css" />
    <jsp:scriptlet>
    session.setAttribute( "EmpList", data.DAO.getData());
    </jsp:scriptlet>
    <h2 align="center">Emp Table with Display tag</h2>
    <display:table name="sessionScope.EmpList" pagesize="4">
    <display:column property="empId" title="ID" />
    <display:column property="empName" title="Name" />
    <display:column property="empJob" title="Job" />
    <display:column property="empSal" title="Salary" />
    </display:table>
    </jsp:root>
    pagingEmp.jsp
  4. Create the Employee class, which is the bean that will hold the Employee data as shown below:
    public class Employee {
    public long empId;
    public String empName;
    public String empJob;
    public long empSal;
    public long getEmpId() {
    return empId;
    }
    public void setEmpId(long empId) {
    this.empId = empId;
    }
    public String getEmpJob() {
    return empJob;
    }
    public void setEmpJob(String empJob) {
    this.empJob = empJob;
    }
    public String getEmpName() {
    return empName;
    }
    public void setEmpName(String empName) {
    this.empName = empName;
    }
    public long getEmpSal() {
    return empSal;
    }
    public void setEmpSal(long empSal) {
    this.empSal = empSal;
    }
    }
    Employee.java
  5. Define the styles for displaying the table. Displaytag renders the tables as simple HTML tables, with the standart tr,td,th and table tags. The css file is shown below
    td {
    font-size: 0.65em;
    font-family: Verdana, Geneva, Arial, Helvetica, sans-serif;
    font-size: 11px;
    }
    th {
    font-size: 0.85em;
    border-top: 2px solid #ddd;
    border-right: 2px solid #ddd;
    border-left: 2px solid #666;
    border-bottom: 2px solid #666;
    }
    table {
    border: 1px dotted #666;
    width: 80%;
    margin: 20px 0 20px 0;
    }
    th,td {
    margin: 0;
    padding: 0;
    text-align: left;
    vertical-align: top;
    background-repeat: no-repeat;
    list-style-type: none;
    }
    thead tr {
    background-color: #bbb;
    }
    tr.odd {
    background-color: #fff;
    }
    tr.even {
    background-color: #ddd;
    }
    screen.css
  6. Configure Hibernate for accessing database
    1. Create the Employee.hbm.xml file to map the Employee bean with the database table as shown below
      <?xml version="1.0"?>
      <!DOCTYPE hibernate-mapping PUBLIC
      "-//Hibernate/Hibernate Mapping DTD 3.0//EN"
      "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
      <hibernate-mapping>
      <class name="beans.Employee" table="Emp">
      <id name="empId" column="EMPNO" type="long">
      <generator class="native"/>
      </id>
      <property name="empName" column="ENAME" />
      <property name="empJob" column="JOB" />
      <property name="empSal" column="SAL" type="long"/>
      </class>
      </hibernate-mapping>
      Employee.hbm.xml

      This file is placed in the same directory as the Employee.java class.
    2. Create the Hibernate Configuration file hibernate.cfg.xml in the root directory of the classes.
      <?xml version='1.0' encoding='utf-8'?>
      <!DOCTYPE hibernate-configuration PUBLIC
      "-//Hibernate/Hibernate Configuration DTD 3.0//EN"
      "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
      <hibernate-configuration>
      <session-factory>
      <property name="connection.driver_class">oracle.jdbc.driver.OracleDriver</property>
      <property name="connection.url">jdbc:oracle:thin:@localhost:1521/orcl</property>
      <property name="connection.username">scott</property>
      <property name="connection.password">tiger</property>
      <property name="dialect">org.hibernate.dialect.Oracle9Dialect</property>
      <mapping resource="beans/Employee.hbm.xml"/>
      <property name="hibernate.current_session_context_class">thread</property>
      </session-factory>
      </hibernate-configuration>
      hibernate.cfg.xml
  7. Create a class for Data access as shown below
    public class DAO {
    public static List getData() {
    SessionFactory sessionFactory = new Configuration().configure().buildSessionFactory();
    Session session = sessionFactory.getCurrentSession();
    List result = null;
    try {
    session.beginTransaction();
    result = session.createQuery("from Employee").list();
    session.getTransaction().commit();
    } catch (Exception e) {
    e.printStackTrace();
    }
    return result;
    }
    }
    DAO.java
Running this example on tomcat shows the Emp table data in a table with 4 records per page, as set in the pagesize attribute of the display:table tag.

Wednesday, November 29, 2006

Google co-op search

I put up a new google co-op search box on top of the blog. I am trying to build a Java search service using Google co-op. I left the custom search engine open to modification. Please feel free to add to it at the Java search homepage.

Tuesday, November 28, 2006

Merge PDF files with iText

New posts with iText 5.5.12

Following are two new posts for PDF Merge with iText 5.5.12 The previous post described the use of iText to create a simple PDF. This post describes the use of iText to merge multiple PDF documents. The sample code here contains a method that will take the a list of PDF files as input, along with an OutputStream to which the merged PDF file will be written. This example also implements paging, i.e. along with merging the PDF files the merged PDF file also adds page numbers to the generated PDF file. The following is a brief description of the steps involved in merging PDF files using iText.
Skip to Sample code
  1. Create a PdfReader for each input PDF file
    PdfReader pdfReader = new PdfReader(pdf);
  2. Create a document object to represent the PDF.
    Document document = new Document();
  3. Create a PdfWriter for the target OutputStream
    PdfWriter writer = PdfWriter.getInstance(document, outputStream);
  4. Select a font with which the page numbers will be written
    BaseFont bf = BaseFont.createFont(BaseFont.HELVETICA, BaseFont.CP1252, BaseFont.NOT_EMBEDDED);
  5. Create a a PdfContentByte object to hold the data of the merged PDF
    PdfContentByte cb = writer.getDirectContent();
  6. Add individual pages from the source to the target.
    document.newPage();
    pageOfCurrentReaderPDF++;
    currentPageNumber++;
    page1 = writer.getImportedPage(pdfReader, pageOfCurrentReaderPDF);
    cb.addTemplate(page1, 0, 0);
  7. Add page text at the bottom of the page
    cb.beginText();
    cb.setFontAndSize(bf, 9);
    cb.showTextAligned(PdfContentByte.ALIGN_CENTER, "Page " + currentPageNumber + " of " + totalPages, 520, 5, 0);
    cb.endText();
The following is the complete source code merging PDF files.
public class MergePDF {
public static void main(String[] args) {
try {
List<InputStream> pdfs = new ArrayList<InputStream>();
pdfs.add(new FileInputStream("c:\\pdf\\2.pdf"));
pdfs.add(new FileInputStream("c:\\pdf\\3.pdf"));
OutputStream output = new FileOutputStream("c:\\pdf\\merge.pdf");
MergePDF.concatPDFs(pdfs, output, true);
} catch (Exception e) {
e.printStackTrace();
}
}

public static void concatPDFs(List<InputStream> streamOfPDFFiles, OutputStream outputStream, boolean paginate) {

Document document = new Document();
try {
List<InputStream> pdfs = streamOfPDFFiles;
List<PdfReader> readers = new ArrayList<PdfReader>();
int totalPages = 0;
Iterator<InputStream> iteratorPDFs = pdfs.iterator();

// Create Readers for the pdfs.
while (iteratorPDFs.hasNext()) {
InputStream pdf = iteratorPDFs.next();
PdfReader pdfReader = new PdfReader(pdf);
readers.add(pdfReader);
totalPages += pdfReader.getNumberOfPages();
}
// Create a writer for the outputstream
PdfWriter writer = PdfWriter.getInstance(document, outputStream);

document.open();
BaseFont bf = BaseFont.createFont(BaseFont.HELVETICA, BaseFont.CP1252, BaseFont.NOT_EMBEDDED);
PdfContentByte cb = writer.getDirectContent(); // Holds the PDF
// data

PdfImportedPage page;
int currentPageNumber = 0;
int pageOfCurrentReaderPDF = 0;
Iterator<PdfReader> iteratorPDFReader = readers.iterator();

// Loop through the PDF files and add to the output.
while (iteratorPDFReader.hasNext()) {
PdfReader pdfReader = iteratorPDFReader.next();

// Create a new page in the target for each source page.
while (pageOfCurrentReaderPDF < pdfReader.getNumberOfPages()) {
document.newPage();
pageOfCurrentReaderPDF++;
currentPageNumber++;
page = writer.getImportedPage(pdfReader, pageOfCurrentReaderPDF);
cb.addTemplate(page, 0, 0);

// Code for pagination.
if (paginate) {
cb.beginText();
cb.setFontAndSize(bf, 9);
cb.showTextAligned(PdfContentByte.ALIGN_CENTER, "" + currentPageNumber + " of " + totalPages, 520, 5, 0);
cb.endText();
}
}
pageOfCurrentReaderPDF = 0;
}
outputStream.flush();
document.close();
outputStream.close();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (document.isOpen())
document.close();
try {
if (outputStream != null)
outputStream.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
}
MergePDF.java

PDF generation with iText

iText is a Java API for reading, writing, and editing PDF documents and PDF forms. The latest release (v1.4.7) adds support for AES encryption of documents, as well as better integration with Acrobat forms. The iText project's PDF library is perhaps the best-known open-source API to create, read, and edit PDF documents from Java. iText supports a rich set of PDF capabilities, such as the ability to select fonts, specify hyphenation, generate PDF in various page sizes, include tables and graphics in PDF documents, print page numbers, headers, and footers, use watermarks, and even supports the creation and reading of barcodes and document encryption. iText API also works with other report-oriented Java projects, such as JFree, allowing easy insertion of JFree reports or JFree charts in PDF documents.
The following example demonstrates how to create a simple PDF form using iText.
  1. Download the latest version of iText (v 1.4.7).
  2. The following piece of code can be used to generate a simple pdf with some text on it
    import java.io.FileOutputStream;
    import java.io.IOException;
    import com.lowagie.text.Document;
    import com.lowagie.text.DocumentException;
    import com.lowagie.text.Paragraph;
    import com.lowagie.text.pdf.PdfWriter;
    public class TestiText {
    public static void main(String[] args) {
    Document document = new Document();
    try {
    PdfWriter.getInstance(document, new FileOutputStream("c:\\test.pdf"));
    document.open();
    document.add(new Paragraph("iText Works :)"));
    } catch (DocumentException de) {
    de.printStackTrace();
    } catch (IOException ioe) {
    ioe.printStackTrace();
    }
    document.close();
    }
    }
    TestiText.java
  3. Paragraph is one of the page elements in iText a listing of the page elements can be found here.

Monday, November 27, 2006

Scheduling with Quartz

Batch solutions are ideal for processing that is time and/or state based:
  • Time-based: The business function executes on a recurring basis, running at pre-determined schedules.
  • State-based: The jobs will be run when the system reaches a specific state.
Batch processes are usually data-centric and are required to handle large volumes of data off-line without affecting your on-line systems. This nature of batch processing requires proper scheduling of jobs. Quartz is a full-featured, open source job scheduling system that can be integrated with, or used along side virtually any Java Enterprise of stand-alone application. The Quartz Scheduler includes many enterprise-class features, such as JTA transactions and clustering. The following is a list of features available:
Skip to Sample Code
  • Can run embedded within another free standing application
  • Can be instantiated within an application server (or servlet container).
  • Can participate in XA transactions, via the use of JobStoreCMT.
  • Can run as a stand-alone program (within its own Java Virtual Machine), to be used via RMI
  • Can be instantiated as a cluster of stand-alone programs (with load-balance and fail-over capabilities)
  • Supoprt for Fail-over
  • Support for Load balancing.
The following example demonstrates the use of Quartz scheduler from a stand-alone application. Follow these steps to setup the example, in Eclipse.
  1. Download the latest version of quartz from opensymphony.
  2. Make sure you have the following in your class path (project-properties->java build path):
    • The quartz jar file (quartz-1.6.0.jar).
    • Commons logging (commons-logging-1.0.4.jar)
    • Commons Collections (commons-collections-3.1.jar)
    • Add any server runtime to your classpath in eclipse. This is for including the Java transaction API used by Quartz. Alternatively, you can include the JTA class files in your classpath as follows
      1. Download the JTA classes zip file from the JTA download page.
      2. Extract the files in the zip file to a subdirectory of your project in Eclipse.
      3. Add the directory to your Java Build Path in the project->preferences, as a class directory.
  3. Implement a Quartz Job: A quartz job is the task that will run at the scheduled time.
    public class SimpleJob implements Job {
    public void execute(JobExecutionContext ctx) throws JobExecutionException {
    System.out.println("Executing at: " + Calendar.getInstance().getTime() + " triggered by: " + ctx.getTrigger().getName());
    }
    }
    SimpleJob.java
  4. The following piece of code can be used to run the job using a scheduler.
    public class QuartzTest {
    public static void main(String[] args) {
    try {
    // Get a scheduler instance.
    SchedulerFactory schedulerFactory = new StdSchedulerFactory();
    Scheduler scheduler = schedulerFactory.getScheduler();

    long ctime = System.currentTimeMillis();

    // Create a trigger.
    JobDetail jobDetail = new JobDetail("Job Detail", "jGroup", SimpleJob.class);
    SimpleTrigger simpleTrigger = new SimpleTrigger("My Trigger", "tGroup");
    simpleTrigger.setStartTime(new Date(ctime));

    // Set the time interval and number of repeats.
    simpleTrigger.setRepeatInterval(100);
    simpleTrigger.setRepeatCount(10);

    // Add trigger and job to Scheduler.
    scheduler.scheduleJob(jobDetail, simpleTrigger);

    // Start the job.
    scheduler.start();
    } catch (SchedulerException ex) {
    ex.printStackTrace();
    }
    }
    }
    QuartzTest.java

    A trigger is used to define the schedule in which to run the job.
For more information on batch processing visit: "High volume transaction processing in J2EE"

Wednesday, November 22, 2006

Java 5 Concurrency

The last few posts described a few of the new concurrency constructs available from Java 5. The reason for this interest in concurrency in Java, inspite of Java EE recommending only simple threading, is that, there is an increasing availability of multicore and multiprocessor machines. The new architectures move toward having lots of cores but low clock speed per core. This might pose some problems, as Billy Newport discusses in his latest post "Multi-core may be bad for Java". The likely problems are
  • Java likely has a longer path length than languages like C and clock speeds won't help with this.
  • JavaEE promotes a simple threading model for applications
  • Garbage collection remains heavily dependant on clock speed for small pauses.
These problems may push the Java development towards heavy multithreading, and light-weight containers. For more details visit the actual post. The following is a list of posts that were made on this blog covering Concurrency.
  • Java: Handling Interrupts: Describes the nuances of interrupting threads and handling the interrupts in a proper way.
  • New features in Concurrency: An overview of the new concurrency features in Java 5.
  • Executors: ThreadPool: A look at Excutors, and an example of how to use ThreadPoolExecutor.
  • Callable and Future: Description and example of how to use Callable and Future interfaces to return values from threads.
  • Locks
  • Reader-Writer Locks: Description of ReaderWriterLock, with a sample implementation of ReentrantReaderWriterLock
  • Conditions: Description and example of how to use Condition interface to have multiple groups of threads waiting to execute a critical section.
  • Synchronizers: Description of various types of synchronizers availabe in Java 5.
  • Selecting Synchronizer: A brief overview of which Synchronizer to use in which situation.
  • Selecting Locks: A brief overview of which Lock to use in which situation.

Java 5 Concurrency: Selecting Locks

As with synchronizers, there is a choice of a few implementations of Locks in Java 5. In the previous post (selecting synchronizers), I gathered a few usage scenarios where the different synchronizers may be used. In this post, I will put together a few usage scenarios where the different Lock implementations may be used.
  • Lock: Lock implementations provide more extensive locking operations than can be obtained using synchronized methods and statements. The standard lock implementation may be used anywhere there is a need to restrict access to a shared resource so that only one thread of execution may access the resource. Spceifically,
    • When acquiring and releasing a lock may happen in different lexical scopes.
    • Chain locking: you acquire the lock of node A, then node B, then release A and acquire C, then release B and acquire D and so on ...
  • Read/Write Lock: maintains a pair of associated locks, one for read-only operations and one for writing. The read lock may be held simultaneously by multiple reader threads, so long as there are no writers. The write lock is exclusive.Usage Scenarios:
    • In scenarios where where is high frequency of reads and the duration of each is sufficiently long. User registries have such a data access pattern.
It should be noted that the standard lock may be used in any place a ReadWriteLock is used, but the performance gains by using a ReadWriteLock will be high when used in the proper application. Only profiling and measurement will establish whether the use of a read-write lock is suitable for your application.

Java 5 Concurrency: Selecting Synchronizers

Few of the recent posts described the use of the various constructs provided by Java 5, through the java.util.concurrent package. The next set describes which scenarios can be solved by the use of each of these constructs. I tried to gather the usage scenarios for the various synchronizers available in Java 5 in this post.
  • Semaphores: A semaphore is the classic method for restricting access to shared resources in a multi-processing environment. While a synchronized block allows only one thread to access a resource, a semaphore allows multiple threads to access a shared resource. Semaphores are often used to restrict the number of threads than can access some resource.
    • Maintaining multiple connections to a database: Define a semaphore, which has same number of permits as there are connections to the database. If all the permits are used, then a thread requesting a connection will be blocked until another thread releases a permit, when this thread may acquire a permit.
    • Binary semaphore can be used in place of any Lock implementation. Such a implementation has an advantage when recovering from deadlocks, since the lock can be unlocked by another thread.
    • When throughput advantages of non-fair ordering often outweigh fairness considerations. Semphores allow barging behaviour. The tryAcquire() method can be used for barging ahead of other threads, irrespective of fairness settings. The tryAcquire(0, TimeUnit.SECONDS) respects fairness setting.
  • Barriers: A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point. Scenrarios:
    • Joins: When you join a set of threads and start a new set, there may be a need for each thread to save state at the join point. A cyclic barrier may be used for such a scenario.
  • Latches: A synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes. Usage Scenarios:
    • Divide a problem into N parts, describe each part with a Runnable that executes that portion and counts down on the latch, and queue all the Runnables to an Executor. When all sub-parts are complete, the coordinating thread will be able to pass through await.
    • Latching works well with initialization tasks, where you want no process to run until everything needed is initialized.
  • Exchangers: A synchronization point at which two threads can exchange objects, the condition being that the exchanging threads have to be paired up, and a specific data type must be exchanged. Usage Scenarios:
    • An Exchanger is often used when you have two threads, one consuming a resource, and the other producing it. Similar to the producer/consumer problem, but where the buffer can be in one of only two states - empty or full.

Tuesday, November 21, 2006

Java 5 Concurrency: Synchronizers

Java 5 introduces general purpose synchronization classes, including semaphores, mutexes, barriers, latches, and exchangers, which facilitate coordination between threads. These classes are a apart of the java.util.concurrent package. A brief description of each of these follows:

Semaphores
A counting semaphore maintains a set of permits. Each acquire() blocks if necessary until a permit is available, and then takes it. Each release() adds a permit, potentially releasing a blocking acquirer. However, no actual permit objects are used; the Semaphore just keeps a count of the number available and acts accordingly.
Semaphores are often used to restrict the number of threads than can access some (physical or logical) resource.


Barrier
A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point. CyclicBarriers are useful in programs involving a fixed sized group of threads that must occasionally wait for each other. The barrier is called cyclic because it can be re-used after the waiting threads are released.

Exchangers
A synchronization point at which two threads can exchange objects. Each thread presents some object on entry to the exchange method, and receives the object presented by the other thread on return.

Latches
A synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes. A CountDownLatch is initialized with a given count. The await methods block until the current count reaches zero due to invocations of the countDown() method, after which all waiting threads are released and any subsequent invocations of await return immediately.

For Sample code and more details about Java 5 synchronizers, see the article "GETTING TO KNOW SYNCHRONIZERS" at java.sun.com.

Java 5 Concurrency: Conditions

As mentioned in the previous post Java 5 Locks, the standard wait, notify and notifyall methods do not allow multiple wait sets per object. The Java 5 condition object by factoring out these methods into distinct objects to give the effect of having multiple wait sets per object. Conditions provide a means for one thread to suspend execution (to "wait") until notified by another thread that some state condition may now be true. The key property that waiting for a condition provides is that it atomically releases the associated lock and suspends the current thread, just like Object.wait. A Condition instance is intrinsically bound to a lock. To obtain a Condition instance for a particular Lock instance use its newCondition() method.
Skip to Sample Code
The Condition interface describes condition variables that may be associated with Locks. These are similar in usage to the implicit monitors accessed using Object.wait(), but offer extended capabilities. In particular, multiple Condition objects may be associated with a single Lock. The following example demonstrates the use of Conditions through an implementation of the Producer/Consumer problem (Bounded buffer problem). This example is similar to the one in the reader-writer locks example
public class Buffer {
ReentrantLock lock = new ReentrantLock();
private String[] names;
private int MAXLENGTH = 10;
final Condition notFull = lock.newCondition();
final Condition notEmpty = lock.newCondition();
private int putPosition, getPosition, count;

public Buffer() {
names = new String[MAXLENGTH];
}

public void put(String str) {
lock.lock();
try {
System.out.println("Writer : Array Size : " + count);
while (count == MAXLENGTH) {
System.out.println("Writer Waiting");
notFull.await();
}
count++;
names[putPosition++] = str;
if(putPosition == MAXLENGTH)
putPosition = 0;
Thread.sleep(100);
notEmpty.signal();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
lock.unlock();
}
}

public void get() {
lock.lock();
try {
System.out.println("Reader : Array Size : " +count);
while (count == 0) {
System.out.println("Reader Waiting");
notEmpty.await();
}
count--;
names[getPosition++] = null;
if(getPosition == MAXLENGTH)
getPosition = 0;
Thread.sleep(100);
notFull.signal();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
lock.unlock();
}
}
}
Buffer.java

Note that in this case, we use the await() method instead of wait() and signal() instead of notify(). This is done to ensure that they are different from the methods of Object.
public class Producer implements Runnable {
Buffer myData;
public Producer(Buffer myData) {
super();
this.myData = myData;
}
public void run() {
for(int i = 0; i < 10; i++) {
myData.put(Thread.currentThread().getName() + " : " + i);
}
}
}
Producer.java
public class Consumer implements Runnable {
Buffer myData;
public void run() {
for(int i = 0; i < 10; i++) {
myData.get();
}
}

public Consumer(Buffer myData) {
super();
this.myData = myData;
}
}
Consumer.java
public class ProducerConsumer {
static int THREADS = 10;

public static void main(String[] args) {
Consumer[] consumers = new Consumer[THREADS];
Producer[] producers = new Producer[THREADS];
Buffer data = new Buffer();
Thread[] threads = new Thread[THREADS * 2];
for (int i = 0; i < THREADS; i++) {
consumers[i] = new Consumer(data);
producers[i] = new Producer(data);
threads[i] = new Thread(consumers[i], "" + i);
threads[i + THREADS] = new Thread(producers[i], "" + i);
}

for (int i = 0; i < THREADS * 2; i++) {
threads[i].start();
}

for (int i = 0; i < THREADS * 2; i++) {
try {
threads[i].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
ProducerConsumer.java

Java 5 Concurrency: Reader-Writer Locks

As long as threads are only reading and not writing to shared data, they can run in parallel without any serious issues. The java.util.concurrent.locks package provides classes that implement this type of locking. The ReadWriteLock interface maintains a pair of associated locks, one for read-only and one for writing. The readLock() may be held simultaneously by multiple reader threads, while the writeLock() is exclusive. While this implementation improves performance when compared to the mutex locks, it also depends on other factors,
  • Requires a multi-processor system
  • The frequency of reads as compared to that of writes. A higher frequency of reads is more suitable.
  • Duration of reads as compared to that of writes. Read duration has to be longer, as short reads mean that the readLock will become an overhead.
  • Contention for the data, i.e. the number of threads that will try to read or write the data at the same time
Skip to Sample Code
The following are a few issues to be considered while creating a ReadWriteLock
  • Whether to grant the read lock or the write lock the priority.
  • Whether readers that request the read lock while a reader is active and a writer is waiting.
  • Whether the locks are reentrant.
  • Can the write lock be downgraded to a read lock without allowing an intervening writer?
  • Can a read lock be upgraded to a write lock, in preference to other waiting readers or writers?
The ReentrantReadWriteLock is an implementation of ReadWriteLock with similar semantics to ReentrantLock. The following is a list of properties of the ReentrantReadWriteLock.
  • Acquisition order: Does not impose a reader or writer preference ordering for lock access.
    • When constructed as fair, threads contend for entry using an approximately arrival-order policy.
    • When the write lock is released either the longest-waiting single writer will be assigned the write lock, or if there is a reader waiting longer than any writer, the set of readers will be assigned the read lock.

    • When constructed as non-fair, the order of entry to the lock need not be in arrival order.
    • if readers are active and a writer enters the lock then no subsequent readers will be granted the read lock until after that writer has acquired and released the write lock.
  • Reentrancy: Allows both readers and writers to reacquire read or write locks in the style of a ReentrantLock. A writer can acquire the read lock - but not vice-versa. If a reader tries to acquire the write lock it will never succeed.
  • Interruption of lock acquisition: The read lock and write lock both support interruption during lock acquisition.
  • Condition support: The write lock provides a Condition implementation that behaves in the same way, with respect to the write lock, as the one provided by ReentrantLock.newCondition(). The read lock does not support a Condition.
  • Instrumentation: Supports methods to determine whether locks are held or contended.
The following is Sample Code on how to use ReentrantReadWriteLock. In the example, a set of readers tries to print out an ArrayList represented by data, and a list of Writers tries to add data in to the list. The producer/consumer problem can be implemented by replacing the readers with consumers and writers with producers. Instead of just reading the list, the consumers will have to remove data from the list.
public class Data {
private List<String> names;
ReentrantReadWriteLock lock = new ReentrantReadWriteLock();

public Data() {
names = new ArrayList<String>();
}

public List<String> getNames() {
return names;
}

public void setNames(List<String> names) {
this.names = names;
}

public void add(String str) {
lock.writeLock().lock();
System.out.println("Writer: Number of threads waiting : " + lock.getQueueLength());

// This will alwas be 1.
System.out.println("Writer: Number of write locks waiting : " + lock.getWriteHoldCount());
names.add(str);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
lock.writeLock().unlock();
}

public void readData() {
lock.readLock().lock();
System.out.println("Reader: Number of threads waiting : " + lock.getQueueLength());
System.out.println("Reader: Number of read locks : " + lock.getReadLockCount());
Iterator<String> iter = names.iterator();
while (iter.hasNext()) {
iter.next();
// System.out.println(iter.next());
}
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
lock.readLock().unlock();
}
}
Data.java
public class ListReader implements Runnable {
Data myData;
public void run() {
for(int i = 0; i < 10; i++) {
myData.readData();
}
}

public ListReader(Data myData) {
super();
this.myData = myData;
}
}
ListReader.java
public class ListWriter implements Runnable {
Data myData;
public ListWriter(Data myData) {
super();
this.myData = myData;
}
public void run() {
for(int i = 0; i < 10; i++) {
myData.add(Thread.currentThread().getName() + " : " + i);
}
}
}
ListWriter.java
  public static void main(String[] args) {
ListReader[] readers = new ListReader[THREADS];
ListWriter[] writers = new ListWriter[THREADS];
Data data = new Data();
Thread[] threads = new Thread[THREADS * 2];
for (int i = 0; i < THREADS; i++) {
readers[i] = new ListReader(data);
writers[i] = new ListWriter(data);
threads[i] = new Thread(readers[i], "" + i);
threads[i + THREADS] = new Thread(writers[i], "" + i);
}

for (int i = 0; i < THREADS * 2; i++) {
threads[i].start();
}

for (int i = 0; i < THREADS * 2; i++) {
try {
threads[i].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}

}
}
ReadWriteLockTest.java

Java 5 Concurrency: Locks

he standard way of obtaining locks till Java 1.4 was by the use of synchronized keyword, while it was simple, it also has a number of limitations:
  • No way to back off from an attempt to acquire a lock that is already held, or to give up after waiting for a specified period of time, or to cancel a lock attempt after an interrupt.
  • No way to alter the semantics of a lock, for example, with respect to reentrancy, read versus write protection, or fairness.
  • The use of synchronized forces all lock acquisition and release to occur in a block-structured way: when multiple locks are acquired they must be released in the opposite order, and all locks must be released in the same lexical scope in which they were acquired.
The java.util.concurrent.locks package provides a high-performance lock implementation, which supports specifying a timeout when attempting to acquire a lock, multiple condition variables per lock, non-lexically scoped locks, and support for interrupting threads which are waiting to acquire a lock.
Skip to Sample Code.
The java.util.concurrent.lock package provides a framework for locking and waiting for conditions that is distinct from built-in synchronization and monitors.

The Lock interface supports locking disciplines that offer different locking semantics (reentrant, fair, etc), and that can be used in non-block-structured contexts including hand-over-hand and lock reordering algorithms. For example, some algorithms for traversing concurrently accessed data structures require the use of "hand-over-hand" or "chain locking": you acquire the lock of node A, then node B, then release A and acquire C, then release B and acquire D and so on. Implementations of the Lock interface enable the use of such techniques by allowing a lock to be acquired and released in different scopes, and allowing multiple locks to be acquired and released in any order.
public interface Lock {
void lock();
void lockInterruptibly() throws InterruptedException;
boolean tryLock();
boolean tryLock(long time, TimeUnit unit) throws InterruptedException;
void unlock();
Condition newCondition();
}
The lock() method waits uninterruptably for the lock to be acquired, while the lockInterruptably() method is interruptable. The tryLock() method tries to obtain the lock at the time of invocation, if not it returns false. The tryLock(time, unit), tries to obtain the lock at the time of invocation and waits till timeout or till it is interrupted. The lock() method is not interruptible, all the others are interruptible.
Since the automatic unlocking feature available with synchronized method is unavailable with the new locking mechanism, we have to explicitly unlock. This has to be done in same way we close the I/O streams or JDBC connections
Lock l = ...;
l.lock();
try {
// access the resource protected by this lock
} finally {
l.unlock();
}
This works with the lock() method, but may throw exceptions when used with tryLock() as demonstrated in the example below.
ReentrantLock is the primary implementation class for the Lock interface. The following sample code demonstrates the use of ReentrantLock.
public class ReentrantThread implements Runnable {
private static final ReentrantLock lock = new ReentrantLock(false);
public static String sharedVariable = "";
public void run() {
for (int i = 0; i < 10; i++) {
try {
if (lock.tryLock(1, TimeUnit.MILLISECONDS)) { // Lock obtained
System.out.println("Lock Obtained by thread : " + Thread.currentThread().getName());
sharedVariable += Thread.currentThread().getName() + " : locked \t" + i + "\n";
} else { // Lock not obtained
sharedVariable += Thread.currentThread().getName() + " : unlocked \t" + i + "\n";
}
Thread.sleep(1);
} catch (Exception e) {
e.printStackTrace();
} finally {
if(lock.isLocked()) {
lock.unlock();
}
}
}
}
}
ReentrantThread.java

Note that in the finally block, lock.isHeldByCurrentThread() was used. This was necessary, since the tryLock() method may exit without obtaining the lock, in this case, trying to unlock the thread will result in an IllegalMonitorStateException.
public class LockTester {
public static void main(String[] args) {
Thread t1 = new Thread(new ReentrantThread(), "thread1");
Thread t2 = new Thread(new ReentrantThread(), "thread2");
Thread t3 = new Thread(new ReentrantThread(), "thread3");
t1.start();
t2.start();
t3.start();
try {
t1.join();
t2.join();
t3.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(ReentrantThread.sharedVariable);
}
}
LockTester.java
The next post discusses the usage of Read/Write Locks.

Monday, November 20, 2006

Java 5 Concurrency: Callable and Future

Till Java 1.4, threads could be implemented by either implementing Runnable or extending Thread. This was quite simple, but had a serious limitation - They have a run method that cannot return values. In order to side-step this, most programmers use side-effects (writing to a file etc.) to mimic returning values to the invoker of the thread. Java 5 introduces the Callable interface, that allows users to return values from a thread. This post describes the Callable and Future interfaces and shows an example of how to use these to interfaces.
Jump to Sample Code
public interface Callable<V> {
V call() throws Exception;
}
The call() method is the entry point into a Callable object, and it's return type is the type parameter set in the Callable object. To implement Callable with no return value, use Callable<void>. Also, note that the call() method throws a checked exception, as compared to the run() method in Runnable which does not throw any exception. The Executors class contains utility methods to convert from other common forms to Callable classes. However, Callable cannot be used in place of a Runnable. Callable objects have to be invoked by ExecutorService. The Executor framework provides the Future interface to allow handling the cancellation and returns of a Callable object.
A Future represents the result of an asynchronous computation.
public interface Future {

//Attempts to cancel execution of this task.
boolean cancel(boolean mayInterruptIfRunning);

boolean isCancelled();

boolean isDone();

// Waits if necessary for the computation to complete,
// and then retrieves its result.
V get() throws InterruptedException, ExecutionException;

// Waits if necessary for at most the given time for the computation
// to complete, and then retrieves its result, if available.
V get(long timeout, TimeUnit unit)
throws InterruptedException, ExecutionException, TimeoutException;
}
The result can be retrieved using method get() when the computation has completed, blocking if necessary until it is ready. If you would like to use a Future for the sake of cancellation but not provide a usable result, you can declare types of the form Future<?> and return null as a result of the underlying task. The following example demonstrates the use of Callable and future. The first CallableImpl class implements the Callable interface, and returns an integer that is sent to it's constructor. The CallableTester class invokes the CallableImpl through an executor.
public class CallableImpl implements Callable<Integer> {

private int myName;
CallableImpl(int i){
myName = i;
}

public Integer call() {
for(int i = 0; i < 10; i++) {
System.out.println("Thread : " + getMyName() + " I is : " + i);
}
return new Integer(getMyName());

}

public int getMyName() {
return myName;
}

public void setMyName(int myName) {
this.myName = myName;
}

}
CallableImpl.java
public class CallableTester {

public static void main(String[] args) {
Callable<Integer> callable = new CallableImpl(2);

ExecutorService executor = new ScheduledThreadPoolExecutor(5);
Future<Integer> future = executor.submit(callable);

try {
System.out.println("Future value: " + future.get());
} catch (Exception e) {
e.printStackTrace();
}
}

}
CallableTester.java

ExecutorService extends Executor to provides method to manage thread termination and methods that can produce a Future for tracking progress of one or more asynchronous tasks. The method submit extends Executor.execute(java.lang.Runnable) to create and return a Future. Methods invokeAny and invokeAll perform the most commonly useful forms of bulk execution, executing a collection of tasks and then waiting for at least one, or all, to complete. For an overview of Executors, visit Java 5 Executors.

Friday, November 17, 2006

Java 5 Executors: ThreadPool

The following is a sample of a daemon that accepts requests and processes them concurrently. The daemon accepts requests and creates one thread to handle one request.
while (true) {
request = acceptRequest();
Runnable requestHandler = new Runnable() {
public void run() {
handleRequest(request);
}
};
new Thread(requestHandler).start();
}
While this is a correct implementation, it has some performance drawbacks.
  • Thread lifecycle overhead: If the requests are frequent and lightweight, the thread creation and teardown may become an overhead.
  • Resource consumption:
    • Active threads consume system resources.
    • Idle threads may occupy a lot of memory.
    • Having too many threads competing for CPU time may add an overhead to processing time.
  • Stability: Unbounded thread creation may end in an OutOfMemoryError. This is because of the limits (imposed by the native platform, JVM invocation parameters etc.) on the number of threads that can be created.
This is where the Java 5 executor framework comes in handy. Executor is the primay abstraction for task execution in Java 5.
public interface Executor {
void execute(Runnable command);
}
The executor provides a standard means of decoupling task submission from task execution. The Executors also provide thread lifecycle support and hooks for adding statistics gathering, application management, and monitoring. Executor is based on the producer-consumer pattern, where activities that submit tasks are producers and the threads that execute tasks are consumers. The following sample code shows how to use a ThreadPool (an implementation of Executor).
int NTHREADS = 100;
Executor exec = Executors.newFixedThreadPool(NTHREADS);
while (true) {
request = acceptRequest();
Runnable requestHandler = new Runnable() {
public void run() {
handleRequest(request);
}
};
exec.execute(requestHandler);
}
In this case, the main thread is the producer and requestHandler is the consumer.
Execution Policies
The various Executor implementations provide different execution policies to be set while executing the tasks. For example, the ThreadPool supports the following policies:
  • newFixedThreadPool: Creates threads as tasks are submitted, up to the maximum pool size, and then attempts to keep the pool size constant.
  • newCachedThreadPool: Can add new threads when demand increases, no bounds on the size of the pool.
  • newSingleThreadExecutor: Single worker thread to process tasks, Guarantees order of execution based on the queue policy (FIFO, LIFO, priority order).
  • newScheduledThreadPool: Fixed-size, supports delayed and periodic task execution.
Executor Lifecycle
An application can be shut down either gracefully or abruptly, or somewhere in-between. Executors provide the ability to be shutdown as abruptly or gracefully. This is addressed by the ExecutorService, which implements Executor and adds a number of methods for lifecycle management (and some utility methods).
public interface ExecutorService extends Executor {

void shutdown();

List<Runnable> shutdownNow();

boolean isShutdown();

boolean isTerminated();

boolean awaitTermination(long timeout, TimeUnit unit)
throws InterruptedException;

<T> Future<T> submit(Callable<T> task);

<T> Future<T> submit(Runnable task, T result);

Future<?> submit(Runnable task);

<T> List<Future<T>> invokeAll(Collection<Callable<T>> tasks)
throws InterruptedException;

<T> List<Future<T>> invokeAll(Collection<Callable<T>> tasks,
long timeout, TimeUnit unit)
throws InterruptedException;

<T> T invokeAny(Collection<Callable<T>> tasks)
throws InterruptedException, ExecutionException;

<T> T invokeAny(Collection<Callable<T>> tasks,
long timeout, TimeUnit unit)
throws InterruptedException, ExecutionException, TimeoutException;

}
The first five methods are for lifecycle management. The following code sample shows how the above methods for lifecycle management may be used
doWork() {
ExecutorService exec = ...;
while (!exec.isShutdown()) {
try {
Request request = acceptRequest();
exec.execute(new Runnable() {
public void run() { handleRequest(conn); }
});
} catch (RejectedExecutionException e) {
e.printStackTrace();
}
}
}

public void stop() { exec.shutdown(); }

void handleRequest(Request request) {
if (isShutdownRequest(req))
stop();
else
handle(req);
}

This post presented a brief overview of the Executor framework. The next post will provide more details into the usage of the executor framework, introducing more classes from the java.util.concurrent package.

Thursday, November 16, 2006

Java 5: New features in Concurrency

Most of the new features in concurrency are implemented in the java.util.concurrent packages. There are also new concurrent data structures in the Java Collections Framework:
  • Lock objects support locking idioms that simplify many concurrent applications.
  • Executors define a high-level API for launching and managing threads. Executor implementations provided by java.util.concurrent provide thread pool management suitable for large-scale applications.
  • Concurrent collections make it easier to manage large collections of data, and can greatly reduce the need for synchronization.
  • Atomic variables have features that minimize synchronization and help avoid memory consistency errors.
LOCK OBJECTS
Lock objects work very much like the implicit locks (monitors) used by synchronized code. As with implicit locks, only one thread can own a Lock object at a time. Lock objects also support a wait/notify mechanism, through their associated Condition objects. All the lock objects are defined in the java.util.concurrent.lock package. The biggest advantage of Lock objects over implicit locks is their ability to back out of an attempt to acquire a lock. The tryLock method backs out if the lock is not available immediately or before a timeout expires (if specified). The lockInterruptibly method backs out if another thread sends an interrupt before the lock is acquired.

EXECUTORS
The java.util.concurrent package defines three executor interfaces:
  • Executor: A simple interface that supports launching new tasks.
  • ExecutorService: A sub-interface of Executor, which adds features that help manage the lifecycle, both of the individual tasks and of the executor itself.
  • ScheduledExecutorService: A sub-interface of ExecutorService, supports future and/or periodic execution of tasks.
CONCURRENT COLLECTIONS
The java.util.concurrent package includes a number of additions to the Java Collections Framework.
  • BlockingQueue defines a first-in-first-out data structure that blocks or times out when you attempt to add to a full queue, or retrieve from an empty queue.
  • ConcurrentMap is a subinterface of java.util.Map that defines useful atomic operations. These operations remove or replace a key-value pair only if the key is present, or add a key-value pair only if the key is absent. The standard general-purpose implementation of ConcurrentMap is ConcurrentHashMap, which is a concurrent analog of HashMap.
  • ConcurrentNavigableMap is a subinterface of ConcurrentMap that supports approximate matches. The standard general-purpose implementation of ConcurrentNavigableMap is ConcurrentSkipListMap, which is a concurrent analog of TreeMap.
ATOMIC VARIABLES
The java.util.concurrent.atomic package defines classes that support atomic operations on single variables (ex. AtomicInteger). All classes have get and set methods that work like reads and writes on volatile variables. The atomic compareAndSet method also has these memory consistency features, as do the simple atomic arithmetic methods that apply to integer atomic variables.

References

Java: Handling Interrupts

An interrupt is an indication to a thread that it should stop what it is doing and do something else. It's up to the programmer to decide exactly how a thread responds to an interrupt, but it is very common for the thread to terminate. A thread can be interrupted by calling the threadObject.interrupt() on the thread object.
threadObject.interrupt();
In order for the interrupt to work, the thread object has to support interruption, i.e. The thread object should check for interruptions periodically, as shown below:
while(!interrupted()) {
doWork();
}
An interrupt does not force the thread to halt (except when the thread is in sleep or wait mode). As shown in the above piece of code, the thread has to check if it is interrupted and take appropriate action (most likely, cleanup and stop execution). There are two ways in which a thread can check if it is interrupted.
  • isInterrupted(): This is a non-static method that simply checks whether a thread is interrupted, and returns true or false.
  • interrupted(): This method (used in the above example) is a static method of the Thread class, which checks if the current thread is interrupted and clears the interrupted state of the thread.

Note: The interrupted state of a thread can be cleaned only by the that thread, no thread can clear the interrupted state of another thread.
While interrupting a thread does not affect it's normal execution (unless the thread is programmed to do so), a thread can also be interrupted by an InterruptedException thrown by the sleep or wait methods. This has to be handled in a proper way, since a thrown exception clears the interrupted state of the thread. InterruptedException is best handled in the following way:
try {
// do some work.
Thread.sleep(sleepTime);
}catch(InterruptedException e) {
Thread.currentThread().interrupt();
}
The question arises, is when and how should interruptions be handled. Here are some general tips handling interruptions:
  • If the thread invokes methods that throw InterruptedException frequently, then it is better to catch the interrupted exception and set the interrupted state of the thread as shown above.
  • If your method blocks, it should respond to interruption, otherwise you must decide what interruption/cancellation means for your method, and make such behavior a part of your method's contract. In general, any method that performs a blocking operation (directly or indirectly), should allow that blocking operation to be cancelled with interrupt and should throw an appropriate exception (as sleep and wait do). If you're using channels, available with the new I/O API introduced in Java 1.4, the blocked thread will get a ClosedByInterruptException exception.
  • Never hide an interrupt by clearing it explicitly or by catching an InterruptedException and continuing normally as it prevents any thread from being cancellable when executing your code.
References

Configuring LDAP in Weblogic

WebLogic Server does not support or certify any particular LDAP servers. Any LDAP v2 or v3 compliant LDAP server should work with WebLogic Server. The LDAP Authentication providers in this release of WebLogic Server (v9.2) are configured to work readily with the SunONE (iPlanet), Active Directory, Open LDAP, and Novell NDS LDAP servers. You can use an LDAP Authentication provider to access other types of LDAP servers. Choose either the LDAP Authentication provider (LDAPAuthenticator) or the existing LDAP provider that most closely matches the new LDAP server and customize the existing configuration to match the directory schema and other attributes for your LDAP server. The server comes with the following Authentication Providers, which help to configure different LDAP servers
  • iPlanet Authentication provider
  • Active Directory Authentication provider
  • Open LDAP Authentication provider
  • Novell Authentication provider
  • generic LDAP Authentication provider
Follow these steps to configure LDAP in Weblogic:
  1. Choose an LDAP Authentication provider that matches your LDAP server and create an instance of the provider in your security realm.
  2. Configure the provider-specific attributes of the LDAP Authentication provider, which you can do through the Administration Console. For each LDAP Authentication provider, there are attributes that:
    1. Enable communication between the LDAP server and the LDAP Authentication provider. For a more secure deployment, BEA recommends using the SSL protocol to protect communications between the LDAP server and WebLogic Server. Enable SSL with the SSLEnabled attribute.
    2. Configure options that control how the LDAP Authentication provider searches the LDAP directory.
    3. Specify where in the LDAP directory structure users are located.
    4. Specify where in the LDAP directory structure groups are located.
    5. Define how members of a group are located.
  3. Configure performance options that control the cache for the LDAP server. Use the Configuration: Provider Specific and Performance pages for the provider in the Administration Console to configure the cache.

FAILOVER

You can configure an LDAP provider to work with multiple LDAP servers and enable failover if one LDAP server is not available. For this, Change the Host attribute in the security_realm > Providers > provider_specific page, to contain a list of hostnames and ports (localhost:389, remotehost:389). When using failover, the Parallel Connect Delay and Connection Timeout attributes have to be set for the LDAP Authentication provider:
  • Parallel Connect Delay—Specifies the number of seconds to delay when making concurrent attempts to connect to multiple servers. An attempt is made to connect to the first server in the list. The next entry in the list is tried only if the attempt to connect to the current host fails. This setting might cause your application to block for an unacceptably long time if a host is down. If the value is greater than 0, another connection setup thread is started after the specified number of delay seconds has passed. If the value is 0, connection attempts are serialized.
  • Connection Timeout—Specifies the maximum number of seconds to wait for the connection to the LDAP server to be established. If the set to 0, there is no maximum time limit and WebLogic Server waits until the TCP/IP layer times out to return a connection failure. Set to a value over 60 seconds depending upon the configuration of TCP/IP.

NOTE
If an LDAP Authentication provider is the only configured Authentication provider for a security realm, you must have the Admin role to boot WebLogic Server and use a user or group in the LDAP directory. You can either create an Administrators group in the LDAP directory, and include your user in that group, or use an existing group and add the group to the admin role in the WebLogic Administration Console. For more information refer to Weblogic documentation: Configuring LDAP providers.

Wednesday, November 15, 2006

Weblogic Security

WebLogic Security Framework provides a simplified application programming interface (API) that can be used by security and application developers to define security services. WebLogic Security Framework also acts as an intermediary between the WebLogic containers (Web and EJB), the Resource containers, and the security providers. In this post, I describe a few concepts and features available in Weblogic Server 9.2 for authentication and authorization and auditing.
Authentication
Authentication is the process of determining whether the client who it claims to be. Generally, authentication is accomplished by the client sending credentials (username/password, certificate, security token etc.) and the server verifying the credentials. WebLogic uses the authentication classes of the Java Authentication and Authorization Service (JAAS), for authentication.
Identity Assertion
Weblogic also supports the concept of perimeter-based authentication, where the actual authentication process occurs at an application perimeter such as a Web server, firewall, or VPN, and outside of WebLogic Server. The perimeter authentication provider then asserts the identity to the Weblogic server using different "Security token types" (e.g., Microsoft Passport, SAML Assertions, or tokens from third-party commercial authentication products). The "security tokens" are validated by the Weblogic Server, which then assigns a username to the token. This username is used by the authentication providers to populate the "Subject" that will be used in authorization. IBM WebSphere implements this by the use of Trust Association Interceptors. WebLogic Server 's support for perimeter-based authentication supports the ability to propagate security tokes over multiple protocols, such as HTTP, and IIOP-CSIv2 (used for EJB layer security).
Authorization
Authorization phase determines if the user has access to the requested application resource. Authorization in Weblogic is divided in to two steps
  1. Decision
  2. Adjudication (Enforce)

Decision
In this step, the WebLogic Security Framework uses the request parameters and user information to determine the roles associated with the user (for this the security framework uses the configured Role Mapping Providers). Based on the user's roles, the Authorization provider determines whether the subject is entitled to access the requested resource i.e the Authorization provider makes the Access Decision. If there are multiple Authorization providers configured, the WebLogic Security Framework delegates the job of reconciling any conflicts in the Access Decisions to the Adjudication provider
Adjudication
The Adjudication provider is required to tally the multiple Access Decisions and render a verdict. The Adjudication provider returns either a TRUE or FALSE verdict to the Authorization providers, which forward it to the resource container through the WebLogic Security Framework.
  • If the decision is TRUE, the resource container dispatches the request to the protected WebLogic resource.
  • If the decision is FALSE, the resource container throws a security exception that indicates that the requestor was not authorized to perform the requested access on the protected WebLogic resource.

Auditing
The auditing process is initiated when a resource container passes a user's authentication information to the WebLogic Security Framework as part of a login request. If, in addition to providing authentication services, the Authentication provider is designed to post audit events, the Authentication provider instantiates an AuditEvent object. The AuditEvent object includes information such as the event type and an audit severity level. The Authentication provider then calls the Auditor Service in the WebLogic Security Framework, passing in the AuditEvent object. The Auditor Service passes the AuditEvent object to the configured Auditing providers' runtime classes, enabling audit event recording. Depending on the Auditing provider implementation, audit records may be written to a file, a database, or some other persistent storage medium.

Tuesday, November 14, 2006

Weblogic: SSO with Windows

An increasing number of intranet-based applications are requiriong Single sign-on (SSO) with between Windows clients (web browser, .NET application etc.) and Java EE servers. The last time, I blogged SSO with IBM WebSphere application server and Windows. To implement this feature, the Microsoft clients must use Windows authentication based on the Simple and Protected Negotiate (SPNEGO) mechanism.

Cross-platform authentication is achieved by emulating the negotiate behavior of native Windows-to-Windows authentication services that use the Kerberos protocol. In order for cross-platform authentication to work, non-Windows servers (WebSphere/WebLogic Servers) need to parse SPNEGO tokens in order to extract Kerberos tokens which are then used for authentication. This post gives a brief overview of the requirements and steps to setup SSO with Windows in Weblogic and provides the resources for further reference:
Requirements
Server
  • Windows 2000 or later installed
  • Fully-configured Active Directory authentication service.
  • WebLogic Server installed and configured properly to authenticate through Kerberos
Client
  • Windows 2000 Professional SP2 or later installed
  • One of the following types of clients:
    • A properly configured Internet Explorer browser. Internet Explorer 6.01 or later is supported.
    • .NET Framework 1.1 and a properly configured Web Service client.
  • Clients must be logged on to a Windows 2000 domain and have Kerberos credentials acquired from the Active Directory server in the domain. Local logons will not work.
Main Steps for Congifuration
Configuring SSO with Microsoft clients requires set-up procedures in the Microsoft Active Directory, the client, and the WebLogic Server domain.
  • Define a principal in Active Directory to represent the WebLogic Server. The Kerberos protocol uses the Active Directory server in the Microsoft domain to store the necessary security information.
  • Any Microsoft client you want to access in the Microsoft domain must be set up to use Windows Integrated authentication, sending a Kerberos ticket when available.
  • In the security realm of the WebLogic Server domain, configure a Negotiate Identity Assertion provider. The Web application or Web Service used in SSO needs to have authentication set in a specific manner. A JAAS login file that defines the location of the Kerberos identification for WebLogic Server must be created.
To configure SSO with Microsoft clients:
  1. Configure your network domain to use Kerberos.
  2. Create a Kerberos identification for WebLogic Server.
    1. Create a user account in the Active Directory for the host on which WebLogic Server is running.
    2. Create a Service Principal Name for this account.
    3. Create a user mapping and keytab file for this account.
  3. Choose a Microsoft client (either a Web Service or a browser) and configure it to use Windows Integrated authentication.
  4. Set up the WebLogic Server domain to use Kerberos authentication.
    1. Create a JAAS login file that points to the Active Directory server in the Microsoft domain and the keytab file created in Step 1.
    2. Configure a Negotiate Identity Assertion provider in the WebLogic Server security realm.
  5. Start WebLogic Server using specific start-up arguments.
References

Monday, November 13, 2006

Opensourcing Java

Java is going to be released in full under the GPL v2. The components of the Java SE implementation that Sun is releasing initially are the Java programming-language compiler (javac) and the Java HotSpot virtual machine. In 2007, Sun will release all unencumbered source-code modules of JDK 6 and JDK 7, along with full build scripts; a few encumbered modules will be released as binary plugins. The code will be available under the GPL v2 license plus the ClassPath Exception. The Java specification will remain under the control of the JCP.

Resources:
  1. James Gosling's blog post open sourcing
  2. The Java source code on java.net
  3. Source Java ME
  4. Java EE source at the Glassfish project

Friday, November 10, 2006

Java EE application on Multi-Core processor based servers

In a new article on onjava.com, titled "Scaling Enterprise Java on 64-bit Multi-Core X86-Based Servers" Michael Juntao Yuan and Dave Jaffe discuss the process of scaling enterprise Java applications on 64-bit multi-core x86 servers. Application developers are playing catch-up with the chip makers such as AMD and Intel as they transition from faster chips to more cores per processor. In the new article, mention that servers such as Dell's PowerEdge 1955 blade which supports up to two 64-bit dual core processors in a blade configuration. The article points out that to take advantage of such systems applications must not only support multiple concurrent threads of execution but also be able to manage the large amounts of memory available to 64-bit processors. The article provides a few useful tips for application developers about to write applications on these machines:
  • Tuning the JVM with Heap and Garbage Collection Arguments that Reflect The Amount of Memory Available.
  • Leveraging Java 5.0 Concurrency Utils to Support Multiple Threads Easier
  • Using new platform APIs such as NIO to Reduce I/O Blocking and the new Logging API (from 1.4)
Apart from these, they also suggest some performance optimizations such as
  • Collapse the tiers which your application runs on. Larger capacity machines may be able to handle both database and appserver tasks for instance reducing items such as network load that would be present with multiple machines.
  • Virtualizing the hardware can also allow administrators to run several JEE application servers in contained environments on the same hardware thus using it more effectively.

Thursday, November 09, 2006

Struts 2: Action Redirects

Struts 2.0 provides the ability to chain multiple actions into a defined sequence or workflow. Action chaining can be achieved in multiple ways:
  • Action Chains: This feature works by applying a test:Chain Result to a given Action, and intercepting its target Action's invocation with a ChainingInterceptor.
  • Redirect After Post: A common a pattern in web application development in which your action will result in a redirect to another action.
    • Redirect Action Result: Uses ActionMapper provided by the ActionMapperFactory to redirect the browser to a URL that invokes the specified action. You can see a simple implementation of this in the struts-blank application. If you look at the example.xml file in the application you will can see that the Login action redirects the result to a different action "Menu".
      <action name="Login!*" method="{1}" class="example.Login">
      <result name="input">/example/Login.jsp</result>
      <result type="redirect-action">Menu</result>
      </action>
      In order to send parameters, use the <param></param> tag, as shown below
      <param name="paramname">value</param>
    • Redirect Result:Calls theHttpServletResponse.sendRedirect(), (a browser redirect). The consequence of doing this means that the action (action instance, action errors, field errors, etc) that was just executed is lost and no longer available. This is because actions are built on a single-thread model. The only way to pass data is through the session or with web parameters (?name=value). In the following code, you can see that the "Menu.Action" is used instead of just Menu.
      <action name="Login!*" method="{1}" class="example.Login">
      <result name="input">/example/Login.jsp</result>
      <result type="redirect">Menu.action</result>
      </action>

Wednesday, November 08, 2006

Oracle: Transparent Data Encryption

Oracle transparent data encryption (available from 10g Release 2) enables you to encrypt database columns and manage encryption keys. Transparent data encryption can be used to protect confidential data such as credit card and social security numbers. An application that processes sensitive data can use this feature to provide strong data encryption with little or no change to the application. Transparent data encryption is a key-based access control system. When a table contains encrypted columns, a single key is used regardless of the number of encrypted columns. The keys for all tables containing encrypted columns are encrypted with the database server master key and stored in a dictionary table in the database. No keys are stored in the clear. Follow these steps to implement encryption on the Database side.
  1. Set the Master Encryption Key by issuing the following command
    ALTER SYSTEM SET ENCRYPTION KEY IDENTIFIED BY password
  2. No database columns can be encrypted wihtout setting the master encryption key first. This command automatically creates an oracle wallet and sets the password for the wallet. The wallet is also opened as a result of this command. Note that there must be a directory $ORACLEBASE/admin/$ORACLESID otherwise you will ge an error
    ORA-28368: cannot auto-create wallet error
  3. Open the wallet: The wallet must be opened explicitly after the database instance starts. When you create the wallet you also open the wallet for operation. After you create the wallet and set the password, every time you open the database, you'll have to open the wallet using the same password as follows:
    alter system set encryption wallet open authenticated by password;
    You can close the wallet like this:
    alter system set encryption wallet close;
    The wallet must be open for Transparent Data Encryption to work. If the wallet is closed, you can access all nonencrypted columns, but not encrypted columns (you will get a "wallet not open" error).
  4. Create a table using CREATE TABLE as shown below
      CREATE TABLE "SCOTT"."ACCOUNT"
    ( "ACCOUNTID" VARCHAR2(40 BYTE),
    "NAME" VARCHAR2(40 BYTE),
    "SSN" VARCHAR2(40 BYTE)
    ) ;
  5. Encrypt the columns: A column can be encrypted by altering the table with the following command
    The default algorithm for encryption is AES with 192-bit key. This can be changed to any other with the "using" clause added after encrypt, as shown below
    alter table accounts modify (ssn encrypt using 'AES128'); 
    Some other encryption algorithms that can be used are AES128, AES192, AES256, or 3DES168.
Do not, use transparent data encryption with these database features:
  • Index types other than B-tree
  • Range scan search through an index
  • Large object datatypes such as BLOB and CLOB
  • Original import/export utilities
  • Other database tools and utilities that directly access data files
For further information refer to Oracle Advanced security administrator's guide.

Tuesday, November 07, 2006

The Java 5 for each loop

A new article on java.net, Nuances of Java 5.0 for-each loop, discusses the Java 5 for-each loop, in great detail. The article covers common programming errors when using the for-each loop, how the enhanced for-loop compares with the equivalent implementation with the regular for-loop, how to write new classes that can be used as targets of a for-each loop compiler optimizations of the for-each loop, and a lot more. Here is a list of the talking points:
  1. The for-each loop can handle only one iteration variable.
  2. Nested iterations are allowed.
  3. Iterating over varargs. Java 5.0 now allows a variable number of arguments of a single type to be passed in as the last parameter to a method. The compiler collects these varargs parameters into an array of that type. The for-each loop can be used to iterated over the varargs.
  4. Do not modify the list during iteration
  5. When appropriate, implement java.util.RandomAccess to allow for compiler optimizations
  6. Consider returning an Iterable rather than a List
  7. Consider returning Iterable views rather than implementing Iterable
  8. Return zero length arrays or empty lists rather than nulls

Popular Posts