How to deploy artifact to the Maven Central Repository

It was quite a long time ago when I wrote the last blog post. Sometimes it just takes time till inspiration finds you… but do not rush forward!
Last year I promised: I will report about growing tomatoes and I keep my word.

Motivation

You have built something niche and you want to give back something to the community, right? Even if the idea (and the implementation) is revolutionary it will not be used by anyone if it is not accessible somehow. To be concrete: if we talk about a jar then this artifact must be available in the Maven Central Repository – or at least in a public repository somewhere. Since Maven Central Repository is the default repo the best is to have our world savior there.

I went through some tutorials but some of them were outdated while others worked only on Linux/Mac. Since I am more a Windows guy I had to walk sometimes in the dark. I do not say that it was difficult but this is something that no one does too frequently thus in a few years when I buy my next notebook I will just scratch my head: what have I done to make it work?

Prerequisites

I assume you have a basic understanding of

  • Windows
  • Git
  • Java
  • Maven

Steps

  1. Create your account on Github
  2. Upload your Maven project and take java-offline-geoip as an example
  3. Study the pom.xml, the important sections are
    • <version>${semver}</version>
    • <distributionManagement>
          <snapshotRepository>
              <id>ossrh</id>
              <url>https://oss.sonatype.org/content/repositories/snapshots</url>
          </snapshotRepository>
          <repository>
              <id>ossrh</id>
              <url>https://oss.sonatype.org/service/local/staging/deploy/maven2</url>
          </repository>
      </distributionManagement>
    • <scm>
          <url>https://github.com/tornaia/java-offline-geoip</url>
          <connection>scm:git:https://github.com/tornaia/java-offline-geoip.git</connection>
          <developerConnection>scm:git:git@github.com:tornaia/java-offline-geoip.git</developerConnection>
      </scm>
    • <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-source-plugin</artifactId>
          <version>${maven-source-plugin.version}</version>
          <executions>
              <execution>
                  <id>attach-sources</id>
                  <goals>
                      <goal>jar</goal>
                  </goals>
              </execution>
          </executions>
      </plugin>
    • <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-javadoc-plugin</artifactId>
          <version>${maven-javadoc-plugin.version}</version>
          <configuration>
              <additionalOptions>
                  <additionalOption>-html5</additionalOption>
              </additionalOptions>
          </configuration>
          <executions>
              <execution>
                  <id>attach-javadocs</id>
                  <goals>
                      <goal>jar</goal>
                  </goals>
              </execution>
          </executions>
      </plugin>
    • <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-gpg-plugin</artifactId>
          <version>${maven-gpg-plugin.version}</version>
          <executions>
              <execution>
                  <id>sign-artifacts</id>
                  <phase>verify</phase>
                  <goals>
                      <goal>sign</goal>
                  </goals>
              </execution>
          </executions>
      </plugin>
    • <plugin>
          <groupId>org.sonatype.plugins</groupId>
          <artifactId>nexus-staging-maven-plugin</artifactId>
          <version>${nexus-staging-maven-plugin.version}</version>
          <extensions>true</extensions>
          <configuration>
              <serverId>ossrh</serverId>
              <nexusUrl>https://oss.sonatype.org</nexusUrl>
              <autoReleaseAfterClose>true</autoReleaseAfterClose>
          </configuration>
      </plugin>
    • <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-scm-plugin</artifactId>
          <version>${maven-scm-plugin.version}</version>
          <configuration>
              <tag>${project.artifactId}-${project.version}</tag>
          </configuration>
      </plugin>
  4. Create a Sonatype account
  5. Open ticket to create your new project and take mine as an example
  6. Install GnuPG
  7. Create a new key pair with Kleopatra: add your name and email too
    • Remember for your passphrase
    • Make a backup of your key pair
    • Generate a revocation certificate – later you need to edit it manually to avoid accidental use
    • Keep these files in a very safe place
  8. Export your certificate to an OpenPGP Keyserver
    1. Configure Kleopatra by right clicking on its system tray icon
    2. Set OpenPGP Keyserver to hkp://pool.sks-keys under Directory Services
    3. Apply
    4. Open Kleopatra by a single click on the system tray icon
    5. Publish it by right clicking on the certificate and selecting Publish on server…
  9. Create/Edit ~/.m2/settings.xml
    • <settings>
        <servers>
          <server>
            <id>ossrh</id>
            <username>YOUR_SONARTYPE_ACCOUNT_NAME</username>
            <password>YOUR_SONARTYPE_ACCOUNT_PASSWORD</password>
          </server>
        </servers>
        <profiles>
          <profile>
            <id>ossrh</id>
            <activation>
              <activeByDefault>true</activeByDefault>
            </activation>
            <properties>
              <gpg.executable>gpg</gpg.executable>
            </properties>
          </profile>
        </profiles>
      </settings>
  10. Deploy your artifact: mvn clean deploy scm:tag -Dsemver=0.0.1
    • It will ask for your certificate’s passphrase
    • After you promoted your first release you need to add a comment to your “new project” ticket then central sync will be activated
    • After you successfully release, your component will be published to Maven Central Repository, typically within 10 minutes, though updates to search.maven.org can take up to two hours
    • If the version ends with -SNAPSHOT then it will go to the snapshots repository – but I know you know Maven very well! 😉

Misc

I was able to deploy snapshots but somehow I had no rights to deploy releases and encountered the following error:
you have no permissions to stage against profile with ID "62d9bff4b07028"? Get to Nexus admin...
I was not the only one with this so I guess it is a common issue but after opening a Jira ticket it just worked.

Large Scale Refactor II. – PoC

After travelling 1.5 hours by public transport from Zürich to Intschi and taking the cable car up to Arnisee the hiking finally started. The top, Sunniggrätli offers not just an amazing view but a small mountain hütte too: I have never had a lunch with such an amazing view. The cover image was taken from the dining table: no filters, no photoshop. At the end of the day we climbed – according to my FitBit Blaze – 296 floors, walked 19.9 Kilometers and burned 5453 calories. We took not just the aching toes home but a lifetime experience too. If you have the opportunity to hike in the Canton of Uri then do not hesitate!

First version

I published what I have so far on GitHub and my mind is really full with TODOs and FIXMEs. At the moment I strongly do not recommend to install it unless you want nothing but play with it.

I called three libraries for help: ShrinkWrap to resolve Maven dependencies, JavaParser and ASM to parse Java source and class files.

Some really basic scenarios are already implemented and covered with automated tests. I had two options: building up an object structure representing the Maven modules then executing the transformation steps on them. The other way is to create real Maven projects: folders and source files on the file system and using them as input and writing the results back to the disk. I voted for the second but implemented a few but real unit tests beside these high level ones.

Jenkins helped me to make the build more environment independent. Some issues immediately popped out: ignoring different line endings (CR, LF) and indentation is now solved.

large-scale-refactor-ui-first-version
I am aware of its ugliness

Next

A more aesthetic UI, much more handled scenarios and figure out how to push LSR to JetBrains IntelliJ IDEA Plugin Repository.

Large Scale Refactor I. – An idea

I am sending out Anmeldeformular für Wohnen, Betreibungsauszug and Kopies auf der Aufenthaltsbewilligung once or sometimes twice each day mostly via e-mail rarely traditional post. Setting up appointments with landlords and actual tenants is my new favorite daily routine. Getting to know the city is just one benefit of searching for a new apartment, however I am now ready to leave behind this joy. Sooner or later it will happen… keep fingers crossed!

The problem

Do you know the single responsibility principle and separation of concerns? Have you ever had a huge and complex module where it – would – took tremendous of time to make the first cut or dependency extraction to simplify it?
Probably after some time you gave up for some reason. Maybe the next super important business feature must be implemented asap… and the code is rotten and more rotten. What if this redesign could be like method renaming? Shift + F6 or Alt + Shift + R!

An example

There is a huge module “core” with many dependencies: logging, persistence, web/servlet, etc…

You want to move all logging related dependencies to a new “core-logging” module. So you select logback-core, logback-classic, slf4j-api, jcl-over-slf4j and with one click they are all moved to a new module as well as all the related classes.

As a next step you will move the hibernate/jpa/db/sql dependencies exactly like this to a new “core-persistence” module.

I know: it is a super easy and trivial example. What about really mean and complex scenarios? Will it work? Is this a good idea at all?

Next

A primitive POC is under development and at first I will create a simple plugin for IntelliJ IDEA around it. It will be publish on GitHub soon!

SonarQube and C#: drill down results of test cases

The sparkling blue of the Adriatic has gone but the brown skin and memory remains. This year we focused on Istria: Pula, Mali Lošinj and Osor. Sailing this region of Croatia and the island hopping with a Hanse 385 Cruiser was a perfect summer adventure.
But the vacation is over and now I am heading from Budapest back to Zürich on the ÖBB Railjet 162: plenty of time to prepare this post!

 

The Question

I cloned https://github.com/SonarSource/sonar-examples.git then opened a command prompt under SonarSource/sonar-examples/tree/master/projects/languages/csharp and ran the following commands (based on Unit Test Execution Results Import (C#, VB.NET)):

MSBuild.SonarQube.Runner.exe begin /k:"org.sonarqube:csharp-simple-sq-scanner-msbuild" /n:"C# :: Simple Project :: SonarQube Scanner for MSBuild" /v:"1.0" /d:sonar.cs.xunit.reportsPaths="%CD%\XUnitResults.xml"
MSBuild.exe /t:Rebuild   
packages\xunit.runner.console.2.1.0\tools\xunit.console.exe XUnitProject1\bin\Debug\XUnitProject1.dll -xml %CD%\XUnitResults.xml
MSBuild.SonarQube.Runner.exe end

Under metric/tests/list I get “Unit Tests 1” and that is OK but beneath it the list is empty:

WWzgb

I simply did not understand why I was not able to drill down the test results of this C# project. I found nothing suspicious in the logs:

INFO: Sensor org.sonar.plugins.csharp.CSharpUnitTestResultsProvider$CSharpUnitTestResultsImportSensor
INFO: Parsing the XUnit Test Results file C:\workspace\SonarSource-sonar-examples-92828b2\projects\languages\csharp\XUnitResults.xml
INFO: Sensor org.sonar.plugins.csharp.CSharpUnitTestResultsProvider$CSharpUnitTestResultsImportSensor (done) | time=15ms

It bothered me so much I asked around on the official SonarQube mailing list too. I got one response… at least I was not alone with the issue. It is more than nothing. I gave Stack Overflow a chance too but no luck so I started to do the dirty job.

 

The Investigation

Comparing the JSON responses

I re-used two very simple projects with the help of SonarSource/sonar-examples: a Java and a C# one. I pushed the results of both to a local SonarQube instance and then compared the JSON responses.

C#:

{  
   "paging":{  
      "pageIndex":1,
      "pageSize":100,
      "total":0
   },
   "baseComponent":{  
      "id":"AVV6fdxDjdEPPoHbjrEx",
      "key":"org.sonarqube:csharp-simple-sq-scanner-msbuild",
      "name":"C#MSBuild",
      "qualifier":"TRK",
      "measures":[  
         {  
            "metric":"tests",
            "value":"1",
            "periods":[  
               {  
                  "index":1,
                  "value":"0"
               },
               {  
                  "index":2,
                  "value":"1"
               },
               {  
                  "index":3,
                  "value":"0"
               }
            ]
         }
      ]
   },
   "components":[  

   ]
}

Java:

{  
   "paging":{  
      "pageIndex":1,
      "pageSize":100,
      "total":1
   },
   "baseComponent":{  
      "id":"AVZADqxkeSE9GOvlogza",
      "key":"org.sonarqube:example-ut-maven-jacoco",
      "name":"JavaUT",
      "qualifier":"TRK",
      "measures":[  
         {  
            "metric":"tests",
            "value":"2",
            "periods":[  
               {  
                  "index":1,
                  "value":"2"
               },
               {  
                  "index":2,
                  "value":"2"
               },
               {  
                  "index":3,
                  "value":"2"
               }
            ]
         }
      ]
   },
   "components":[  
      {  
         "id":"AVZADq5p6Qz7lWVR41QW",
         "key":"org.sonarqube:example-ut-maven-jacoco:src/test/java/example",
         "name":"src/test/java/example",
         "qualifier":"DIR",
         "path":"src/test/java/example",
         "measures":[  
            {  
               "metric":"tests",
               "value":"2",
               "periods":[  
                  {  
                     "index":1,
                     "value":"2"
                  },
                  {  
                     "index":2,
                     "value":"2"
                  },
                  {  
                     "index":3,
                     "value":"2"
                  }
               ]
            }
         ]
      }
   ]
}

Wrong way. The data is just not there thus it can not be a UI bug.

 

Debugging the backend

I cloned the Git project of SonarQube and opened it with IntelliJ IDEA.

Starting the Sonar instance in debug was easy:

%SONAR%/conf/wrapper.conf:

wrapper.java.additional.1=-Djava.awt.headless=true -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=1044

%SONAR%/conf/sonar.properties:

sonar.web.javaAdditionalOpts=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=1045

After debugging the pits of hell I got the feeling that the backend is somehow OK but the required data is just not there.

 

Analyzing the C# plugin

What about SonarQube’s C# plugin? I read the docs and found something:

Drilldown on Test Execution Results is not supported
Tests execution results will be displayed on project level dashboards.

Gotcha! From this point the events sped up:

It is not a bug or configuration issue. I posted my findings to the Stack Overflow and to the mailing list too.

 

Next?

I keep my eyes open and watching these open tickets and mailing lists.

 

Update

  • SONARCS-657 – Import test cases and attach them to their corresponding test source files (allow to drill down) – Closed and Won’t fix