09 September, 2017

Matrix animation

This weekend's I had a bit of time and I've added ability to animate matrices in LED Matrix Editor.
Also I've added color selector and refined design:



Previous version is available as well

31 August, 2017

Spotfire Internals

SBDF and STDF Reader

SBDF and STDF Reader

Spotfire can read table data from different type of files.

But there are two native Spotfire formats:

  • STDF (Spotfire Text Data File) format
  • SBDF (Spotfire Binary Data File) format

Both supported by the Rocket Table

Rocket Table is a lightweight table viewer for files in SAS7BDAT, SBDF, STDF and CSV formats. It supports sime nice features like: filtering, sorting, highlighting. It is cross-platform and minimalistic.

So if you have *.sbdf or *.stdf files, but don't have Spotfire, you are able to read these files using this application.

Rocket Table

See more about Spotfire internals on GitHub

Xantorohara, 2017-08-31, Spotfire

Create, get, remove datatable relations in Spotfire using Python

Create, get, remove datatable relations in Spotfire using Python

Imagine you have a data table with countries:

Country code Country name
HKG Hong Kong
ISR Israel
MYS Malaysia

And data table with languages related to these countries:

Country code Language Percentage
HKG Canton Chinese 88.7
HKG Chiu chau 1.4
HKG English 2.2
HKG Fukien 1.9
HKG Hakka 1.6
ISR Arabic 18.0
ISR Hebrew 63.1
ISR Russian 8.9
MYS Chinese 9.0
MYS Dusun 1.1
MYS English 1.6
MYS Iban 2.8
MYS Malay 58.4
MYS Tamil 3.9

Where "Country code" is the common field.

You would like to create link between these tables. It is possible to create relation manually in Spotfire, but for some reason you need to create it automatically via Python script.

Create relation between tables

def add_relation(table1, table2, link_expression):
    Document.Data.Relations.Add(
        Document.Data.Tables[table1],
        Document.Data.Tables[table2],
        link_expression)

Usage:

add_relation("country", "lang", "[lang].[Country code]=[country].[Country code]")

Now you are able to select country in the "country" table and all related languages from the "lang" table will also be selected.

Like here:

Spotfire linked tables

Get relation expression

def get_relation(table1, table2):
    return Document.Data.Relations.FindRelation(
        Document.Data.Tables[table1],
        Document.Data.Tables[table2])

Usage:

print(get_relation("country", "lang").Expression)
# [lang].[Country code]=[country].[Country code]

Delete relation

def del_relation(table1, table2):
    relation = Document.Data.Relations.FindRelation(
        Document.Data.Tables[table1],
        Document.Data.Tables[table2])
    if relation:
        Document.Data.Relations.Remove(relation)

Usage:

del_relation("country", "lang")

See more about Spotfire internals on GitHub

Xantorohara, 2017-08-31, Spotfire 7.8.0

30 August, 2017

Check that Spotfire document has a Visualization on a Page using Python

Check that Spotfire document has a Visualization on a Page using Python

Just use this simple Python function:

def hasVisualization(pageName, visName):
    for page in Document.Pages:
        if page.Title == pageName:
            for vis in page.Visuals:
                if vis.Title == visName:
                    return True

Try it:

print(hasVisualization("SomePage", "SomeChart")) # True
print(hasVisualization("SomePage", "ChartNotExists")) # None

Spotfire Visualization

See more about Spotfire internals on GitHub

Xantorohara, 2017-08-30, Spotfire 7.8.0

Check that Spotfire document has a Page using Python

Check that Spotfire document has a Page using Python

Just use this simple Python function:

def hasPage(pageName):
    for page in Document.Pages:
        if page.Title == pageName:
            return True

Try it:

print(hasPage("Page")) # True
print(hasPage("Page (2)")) # True
print(hasPage("SomePage")) # True
print(hasPage("PageNotExists")) # None

Spotfire Pages

See more about Spotfire internals on GitHub

Xantorohara, 2017-08-30, Spotfire 7.8.0

Show URLs in Spotfire Table

Show URLs in Spotfire Table

Spotfire uses auto-detection to set URL renderer for a table column. So, if all values in a column start with "http://" or "https://" it sets URL renderer.

Like here:

Spotfire table with URLs

Moreover, it is possible to set URL renderer for columns in which the values do not look like URLs. But these values can be used as parameters for URLs.

Like here in the "Param Links" column:

Spotfire table parameterized URLs

The only step you need is to specify URL with a placeholder:

Spotfire URL renderer settings

via "Table Properties->Columns"

Spotfire column properties

Don't forget to choose "Link" as a "Renderer".

See more about Spotfire internals on GitHub

Xantorohara, 2017-08-30, Spotfire 7.8.0

29 August, 2017

Show images in Spotfire Table

Show images in Spotfire Table

Imagine that you have a table like this:

Service Name Service Id
Slack slackhq
Travis CI travis-ci
Zenhubio zenhubio
Atom atom

And you would like to see icon images in the "Service Id" column.

You know there is a storage with images, with URLs like this:

https://assets-cdn.github.com/images/modules/site/integrators/${Service Id}.png

So, actually these images should be used instead of appropriate "Service Ids":

https://assets-cdn.github.com/images/modules/site/integrators/slackhq.png
https://assets-cdn.github.com/images/modules/site/integrators/travis-ci.png
https://assets-cdn.github.com/images/modules/site/integrators/zenhubio.png
https://assets-cdn.github.com/images/modules/site/integrators/atom.png

It is easy to achieve it in the Spotfire:

  • Open "Table Properties" -> "Columns"
  • Select column
  • Set "Renderer" type as "Image from URL"
  • Click "Settings..." and put your URL with {$} as a placeholder

Spotfire screenshot

Now you have pretty images in Spotfire Table

See more on GitHub

Xantorohara, 2017-08-29, Spotfire 7.8.0

20 August, 2017

Spring Boot banner

Spring Boot banner

Fun motivates people. And Spring Boot has something for fun - banner!

Love to see this:

Banner screenshot

Look at "Customizing the Banner" in the Spring Boot Reference.

Craft your ASCII banner using these online resouces:

Checkout sample sources from the Spring Field. See also GitHub page.

Spring Boot 1.5.6

17 June, 2017

SBDF and STDF table viewer

As I mentioned in previous post - Rocket table supports not only SAS (.sas7bdat) files. It also supports SBDF (Spotfire Binary Data File) and STDF (Spotfire Text Data File) file formats. Now you do not have to have expensive Spotfire installed in your system. Just use Rocket table it is pretty fast and compact.

SAS table viewer

Couple of years ago I implemented viewer for SAS (.sas7bdat) files. It was SAS Table Explorer. Actually now it is not only viewer for SAS files, but it also supports other table formats. I decided to rename that application to Rocket Table and continue development under this new project.

27 May, 2017

Datasheets

I'm sharing datasheets related I have: http://xantorohara.github.io/datasheets They are related to Arduino boards, modules for Arduino and other hardware components.

06 February, 2017

Spring Boot plugin duplicates dependencies

Recently I found that Spring Boot Maven plugin duplicates some dependencies in the output war file. The problem with time-stamped snapshot dependencies (they have versions like "1.1-20170206.160055-1"). In this case Spring Boot plugin puts both "1.1-SNAPSHOT" and "1.1-20170206.160055-1" versions of jars. Like here:

demo-webapp-1.1-20170206.160055-1.war:\WEB-INF\lib\
                        demo-core-1.1-SNAPSHOT.jar 
                        demo-core-1.1-20170206.160055-1.jar 

This is not critical (application will work), but unpleasant.

The problem occurs with a Spring Boot Maven plugin version before 1.4.4. With 1.4.4 it works fine. So just switch plugin to the 1.4.4, even if you use are using an earlier version of Spring Boot.

<plugins>
    <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <version>1.4.4.RELEASE</version>
    </plugin>
...

After that it will keep only one file per dependency:

demo-webapp-1.1-20170206.160055-1.war:\WEB-INF\lib\
                        demo-core-1.1-SNAPSHOT.jar 

04 February, 2017

Install Arch linux in the VirtualBox

Precondition

  • you have boot into the newly created virtual machine from the Arch live iso

Make new single partition using whole disk space

parted --script /dev/sda mklabel msdos mkpart primary 0% 100% set 1 boot on print
mkfs.ext4 /dev/sda1

Choose a fast mirror

Edit /etc/pacman.d/mirrorlist file, move your preferred server to the top of the file

Install base system

mount /dev/sda1 /mnt
pacstrap /mnt base
genfstab -p /mnt > /mnt/etc/fstab

Configure base system

arch-chroot /mnt
echo en_US.UTF-8 UTF-8 >/etc/locale.gen
localegen
echo LANG=en_US.UTF-8 >/etc/locale.conf
echo a1 >/etc/hostname
ln -s /usr/share/zoneinfo/UTC /etc/localtime
mkinitcpio -p linux
packman -S grub sudo mc bash-completion openssh

Configure network

ip link
systemctl enable dhcpcd@enp0s3.service
systemctl enable sshd.service

Create a user

useradd -m -g users -G wheel -s /bin/bash username
passwd username

Install boot loader

grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sda

Reboot

exit
umount /mnt
reboot

Enjoy your new Arch

Get Spring Boot sources

Spring Boot sources

Very often it is helpful for me to look into the Spring sources in order to understand how some internals work. Of course it is pretty simple to automatically download and view sources using IDE (like IntelliJ IDEA). But sometimes I need just plain source files.

I get these files using "Maven Dependency Plugin" and this pom.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>io.github.xantorohara.springfield</groupId>
    <artifactId>spring-boot-sources</artifactId>
    <version>0.1</version>
    <packaging>pom</packaging>
    <name>spring-boot-sources</name>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.6.RELEASE</version>
        <relativePath/>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <!-- Spring Boot and Spring Framework libraries -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-cache</artifactId>
        </dependency>

        <!-- My current needs -->
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>1.2.0</version>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>3.0.0</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>unpack-dependencies</goal>
                        </goals>
                        <configuration>
                            <classifier>sources</classifier>
                            <outputDirectory>target</outputDirectory>
                            <includeGroupIds>
                                org.springframework,
                                org.springframework.boot,
                                org.mybatis,
                                org.mybatis.spring.boot
                            </includeGroupIds>
                            <useSubDirectoryPerArtifact>true</useSubDirectoryPerArtifact>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

Download source code into the target directory using this command:

mvn process-sources

Just add another dependencies if you need.

Checkout sample sources from the Spring Field. See also GitHub page.

Spring Boot 1.5.6

Oracle LOB storage in row

Do you know how Oracle stores LOB data with "ENABLE STORAGE IN ROW" and "COMPRESS"/"NOCOMPRESS" options?

According to the Oracle documentation:

The maximum amount of LOB data stored in the row is the maximum VARCHAR2 size (4000). This includes the control information as well as the LOB value. If you indicate that the LOB should be stored in the row, once the LOB value and control information is larger than approximately 4000, then the LOB value is automatically moved out of the row.

Create test tablespaces and table:

-- DROP TABLESPACE XTEST_DAT INCLUDING CONTENTS AND DATAFILES;
-- DROP TABLESPACE XTEST_LOB INCLUDING CONTENTS AND DATAFILES;

CREATE TABLESPACE XTEST_DAT DATAFILE '/oracle_tablespaces/xtest_dat.dat' SIZE 10M AUTOEXTEND ON;
CREATE TABLESPACE XTEST_LOB DATAFILE '/oracle_tablespaces/xtest_lob.dat' SIZE 10M AUTOEXTEND ON;

CREATE TABLE xtest_text
(
  text CLOB
)
TABLESPACE XTEST_DAT
LOB (text) STORE AS SECUREFILE (
TABLESPACE XTEST_LOB ENABLE STORAGE IN ROW CHUNK 8192
NOCACHE LOGGING NOCOMPRESS KEEP_DUPLICATES
);

SQL for retrieving tablespaces size:

SELECT TABLESPACE_NAME, SUM(BYTES) BYTES
FROM USER_SEGMENTS
WHERE TABLESPACE_NAME LIKE 'XTEST_%'
GROUP BY TABLESPACE_NAME;

Rows generator:

-- INSERT INTO xtest_text VALUES ('1');
-- COMMIT;

DECLARE
  string CLOB;
  multiplier NUMBER:=1;
BEGIN
  FOR i IN 1..multiplier
  LOOP
    string:= string || dbms_random.string('A', 1000);
  END LOOP;

  FOR i IN 1..1000
  LOOP
    INSERT INTO xtest_text VALUES (string);
  END LOOP;
  COMMIT;
END;

Follow these steps:

  • run with multiplier = 1 to fill table with 1000 rows size of 1000 bytes
  • check tablespaces size
  • run with multiplier = 10 to fill table with 1000 rows size of 10000 bytes
  • check tablespaces size
  • run with multiplier = 5 to fill table with 1000 rows size of 5000 bytes
  • check tablespaces size

Actual results:

With NOCOMPRESS option:

Tablespace name Tablespace size Tablespace size Tablespace size Tablespace size
Inserted 1 row sizeof 1 byte + 1000 rows size of 1K + 1000 rows size of 10K + 1000 rows size of 5K
XTEST_LOB 196608 196608 20185088 27525120
XTEST_DAT 65536 2097152 2097152 2097152

All records size of 1K (with size less than 4000 bytes) are stored in the DAT tablespace. All records size of 10K and 5K (with size more than 4000 bytes) are in the LOB tablespace.

With COMPRESS option:

Tablespace name Tablespace size Tablespace size Tablespace size Tablespace size
Inserted 1 row sizeof 1 byte + 1000 rows size of 1K + 1000 rows size of 10K + 1000 rows size of 5K
XTEST_LOB 196608 196608 10747904 10747904
XTEST_DAT 65536 1048576 2097152 9437184

All records size of 1K (with size less than 4000 bytes) are stored in the DAT tablespace. All records size of 10K (with compressed size more than 4000 bytes) are in the LOB tablespace and partially in DAT. All records size of 5K (with compressed size less than 4000 bytes) are stored in the DAT tablespace.