Matrix animation

This weekend's I had a bit of time and I've added ability to animate matrices in LED Matrix Editor.
Also I've added color selector and refined design:



Previous version is available as well

Create, get, remove datatable relations in Spotfire using Python

Create, get, remove datatable relations in Spotfire using Python

Imagine you have a data table with countries:

Country code	Country name
HKG	Hong Kong
ISR	Israel
MYS	Malaysia

And data table with languages related to these countries:

Country code	Language	Percentage
HKG	Canton Chinese	88.7
HKG	Chiu chau	1.4
HKG	English	2.2
HKG	Fukien	1.9
HKG	Hakka	1.6
ISR	Arabic	18.0
ISR	Hebrew	63.1
ISR	Russian	8.9
MYS	Chinese	9.0
MYS	Dusun	1.1
MYS	English	1.6
MYS	Iban	2.8
MYS	Malay	58.4
MYS	Tamil	3.9

Where "Country code" is the common field.

You would like to create link between these tables. It is possible to create relation manually in Spotfire, but for some reason you need to create it automatically via Python script.

Create relation between tables

def add_relation(table1, table2, link_expression):
    Document.Data.Relations.Add(
        Document.Data.Tables[table1],
        Document.Data.Tables[table2],
        link_expression)

Usage:

add_relation("country", "lang", "[lang].[Country code]=[country].[Country code]")

Now you are able to select country in the "country" table and all related languages from the "lang" table will also be selected.

Like here:

Get relation expression

def get_relation(table1, table2):
    return Document.Data.Relations.FindRelation(
        Document.Data.Tables[table1],
        Document.Data.Tables[table2])

Usage:

print(get_relation("country", "lang").Expression)
# [lang].[Country code]=[country].[Country code]

Delete relation

def del_relation(table1, table2):
    relation = Document.Data.Relations.FindRelation(
        Document.Data.Tables[table1],
        Document.Data.Tables[table2])
    if relation:
        Document.Data.Relations.Remove(relation)

Usage:

del_relation("country", "lang")

See more about Spotfire on GitHub

Xantorohara, 2017-08-31, Spotfire 7.8.0

Check that Spotfire document has a Visualization on a Page using Python

Check that Spotfire document has a Visualization on a Page using Python

Just use this simple Python function:

def has_visualization(page_name, vis_name):
    for page in Document.Pages:
        if page.Title == page_name:
            for vis in page.Visuals:
                if vis.Title == vis_name:
                    return True

Try it:

print(has_visualization("SomePage", "SomeChart")) # True
print(has_visualization("SomePage", "ChartNotExists")) # None

See more about Spotfire on GitHub

Xantorohara, 2017-08-30, Spotfire 7.8.0

Check that Spotfire document has a Page using Python

Check that Spotfire document has a Page using Python

Just use this simple Python function:

def has_page(page_name):
    for page in Document.Pages:
        if page.Title == page_name:
            return True

Try it:

print(has_page("Page")) # True
print(has_page("Page (2)")) # True
print(has_page("SomePage")) # True
print(has_page("PageNotExists")) # None

See more about Spotfire on GitHub

Xantorohara, 2017-08-30, Spotfire 7.8.0

Show URLs in Spotfire Table

Show URLs in Spotfire Table

Spotfire uses auto-detection to set URL renderer for a table column. So, if all values in a column start with "http://" or "https://" it sets URL renderer.

Like here:

Moreover, it is possible to set URL renderer for columns in which the values do not look like URLs. But these values can be used as parameters for URLs.

Like here in the "Param Links" column:

The only step you need is to specify URL with a placeholder:

via "Table Properties->Columns"

Don't forget to choose "Link" as a "Renderer".

See more about Spotfire on GitHub

Xantorohara, 2017-08-30, Spotfire 7.8.0

Show images in Spotfire Table

Show images in Spotfire Table

Imagine that you have a table like this:

Service Name	Service Id
Slack	slackhq
Travis CI	travis-ci
Zenhubio	zenhubio
Atom	atom

And you would like to see icon images in the "Service Id" column.

You know there is a storage with images, with URLs like this:

https://assets-cdn.github.com/images/modules/site/integrators/${Service Id}.png

So, actually these images should be used instead of appropriate "Service Ids":

https://assets-cdn.github.com/images/modules/site/integrators/slackhq.png
https://assets-cdn.github.com/images/modules/site/integrators/travis-ci.png
https://assets-cdn.github.com/images/modules/site/integrators/zenhubio.png
https://assets-cdn.github.com/images/modules/site/integrators/atom.png

It is easy to achieve it in the Spotfire:

• Open "Table Properties" -> "Columns"
• Select column
• Set "Renderer" type as "Image from URL"
• Click "Settings..." and put your URL with {$} as a placeholder

Now you have pretty images in Spotfire Table

See more on GitHub

Xantorohara, 2017-08-29, Spotfire 7.8.0

SBDF and STDF table viewer

As I mentioned in previous post - Rocket table supports not only SAS (.sas7bdat) files. It also supports SBDF (Spotfire Binary Data File) and STDF (Spotfire Text Data File) file formats. Now you do not have to have expensive Spotfire installed in your system. Just use Rocket table it is pretty fast and compact.

SAS table viewer

Couple of years ago I implemented viewer for SAS (.sas7bdat) files. It was SAS Table Explorer. Actually now it is not only viewer for SAS files, but it also supports other table formats. I decided to rename that application to Rocket Table and continue development under this new project.

Spring Boot plugin duplicates dependencies

Recently I found that Spring Boot Maven plugin duplicates some dependencies in the output war file. The problem with time-stamped snapshot dependencies (they have versions like "1.1-20170206.160055-1"). In this case Spring Boot plugin puts both "1.1-SNAPSHOT" and "1.1-20170206.160055-1" versions of jars. Like here:

demo-webapp-1.1-20170206.160055-1.war:\WEB-INF\lib\
                        demo-core-1.1-SNAPSHOT.jar 
                        demo-core-1.1-20170206.160055-1.jar 

This is not critical (application will work), but unpleasant.

The problem occurs with a Spring Boot Maven plugin version before 1.4.4. With 1.4.4 it works fine. So just switch plugin to the 1.4.4, even if you use are using an earlier version of Spring Boot.

<plugins>
    <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <version>1.4.4.RELEASE</version>
    </plugin>
...

After that it will keep only one file per dependency:

demo-webapp-1.1-20170206.160055-1.war:\WEB-INF\lib\
                        demo-core-1.1-SNAPSHOT.jar 

Install Arch linux in the VirtualBox

Precondition

• you have boot into the newly created virtual machine from the Arch live iso

Make new single partition using whole disk space

parted --script /dev/sda mklabel msdos mkpart primary 0% 100% set 1 boot on print
mkfs.ext4 /dev/sda1

Choose a fast mirror

Edit /etc/pacman.d/mirrorlist file, move your preferred server to the top of the file

Install base system

mount /dev/sda1 /mnt
pacstrap /mnt base
genfstab -p /mnt > /mnt/etc/fstab

Configure base system

arch-chroot /mnt
echo en_US.UTF-8 UTF-8 >/etc/locale.gen
localegen
echo LANG=en_US.UTF-8 >/etc/locale.conf
echo a1 >/etc/hostname
ln -s /usr/share/zoneinfo/UTC /etc/localtime
mkinitcpio -p linux
packman -S grub sudo mc bash-completion openssh

Configure network

ip link
systemctl enable dhcpcd@enp0s3.service
systemctl enable sshd.service

Create a user

useradd -m -g users -G wheel -s /bin/bash username
passwd username

Install boot loader

grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sda

Reboot

exit
umount /mnt
reboot

Enjoy your new Arch

Get Spring Boot sources

Spring Boot sources

Very often it is helpful for me to look into the Spring sources in order to understand how some internals work. Of course it is pretty simple to automatically download and view sources using IDE (like IntelliJ IDEA). But sometimes I need just plain source files.
I get these files using "Maven Dependency Plugin" and this pom.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>io.github.xantorohara.springfield</groupId>
    <artifactId>spring-boot-sources</artifactId>
    <version>0.1</version>
    <packaging>pom</packaging>
    <name>spring-boot-sources</name>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.6.RELEASE</version>
        <relativePath/>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <!-- Spring Boot and Spring Framework libraries -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-cache</artifactId>
        </dependency>

        <!-- My current needs -->
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>1.2.0</version>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>3.0.0</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>unpack-dependencies</goal>
                        </goals>
                        <configuration>
                            <classifier>sources</classifier>
                            <outputDirectory>target</outputDirectory>
                            <includeGroupIds>
                                org.springframework,
                                org.springframework.boot,
                                org.mybatis,
                                org.mybatis.spring.boot
                            </includeGroupIds>
                            <useSubDirectoryPerArtifact>true</useSubDirectoryPerArtifact>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

Download source code into the target directory using this command:

mvn process-sources

Just add another dependencies if you need.

Oracle LOB storage in row

Do you know how Oracle stores LOB data with "ENABLE STORAGE IN ROW" and "COMPRESS"/"NOCOMPRESS" options?

According to the Oracle documentation:

The maximum amount of LOB data stored in the row is the maximum VARCHAR2 size (4000). This includes the control information as well as the LOB value. If you indicate that the LOB should be stored in the row, once the LOB value and control information is larger than approximately 4000, then the LOB value is automatically moved out of the row.

Create test tablespaces and table:

-- DROP TABLESPACE XTEST_DAT INCLUDING CONTENTS AND DATAFILES;
-- DROP TABLESPACE XTEST_LOB INCLUDING CONTENTS AND DATAFILES;

CREATE TABLESPACE XTEST_DAT DATAFILE '/oracle_tablespaces/xtest_dat.dat' SIZE 10M AUTOEXTEND ON;
CREATE TABLESPACE XTEST_LOB DATAFILE '/oracle_tablespaces/xtest_lob.dat' SIZE 10M AUTOEXTEND ON;

CREATE TABLE xtest_text
(
  text CLOB
)
TABLESPACE XTEST_DAT
LOB (text) STORE AS SECUREFILE (
TABLESPACE XTEST_LOB ENABLE STORAGE IN ROW CHUNK 8192
NOCACHE LOGGING NOCOMPRESS KEEP_DUPLICATES
);

SQL for retrieving tablespaces size:

SELECT TABLESPACE_NAME, SUM(BYTES) BYTES
FROM USER_SEGMENTS
WHERE TABLESPACE_NAME LIKE 'XTEST_%'
GROUP BY TABLESPACE_NAME;

Rows generator:

-- INSERT INTO xtest_text VALUES ('1');
-- COMMIT;

DECLARE
  string CLOB;
  multiplier NUMBER:=1;
BEGIN
  FOR i IN 1..multiplier
  LOOP
    string:= string || dbms_random.string('A', 1000);
  END LOOP;

  FOR i IN 1..1000
  LOOP
    INSERT INTO xtest_text VALUES (string);
  END LOOP;
  COMMIT;
END;

Follow these steps:

• run with multiplier = 1 to fill table with 1000 rows size of 1000 bytes
• check tablespaces size
• run with multiplier = 10 to fill table with 1000 rows size of 10000 bytes
• check tablespaces size
• run with multiplier = 5 to fill table with 1000 rows size of 5000 bytes
• check tablespaces size

Actual results:

With NOCOMPRESS option:

Tablespace name	Tablespace size	Tablespace size	Tablespace size	Tablespace size
	Inserted 1 row sizeof 1 byte	+ 1000 rows size of 1K	+ 1000 rows size of 10K	+ 1000 rows size of 5K
XTEST_LOB	196608	196608	20185088	27525120
XTEST_DAT	65536	2097152	2097152	2097152

All records size of 1K (with size less than 4000 bytes) are stored in the DAT tablespace. All records size of 10K and 5K (with size more than 4000 bytes) are in the LOB tablespace.

With COMPRESS option:

Tablespace name	Tablespace size	Tablespace size	Tablespace size	Tablespace size
	Inserted 1 row sizeof 1 byte	+ 1000 rows size of 1K	+ 1000 rows size of 10K	+ 1000 rows size of 5K
XTEST_LOB	196608	196608	10747904	10747904
XTEST_DAT	65536	1048576	2097152	9437184

All records size of 1K (with size less than 4000 bytes) are stored in the DAT tablespace. All records size of 10K (with compressed size more than 4000 bytes) are in the LOB tablespace and partially in DAT. All records size of 5K (with compressed size less than 4000 bytes) are stored in the DAT tablespace.