gmes_petap.pl fails, segmentation fault in gmhmme3

[peterjc]

Are you using the latest release?

Yes, 1.8.17 installed with conda.

Describe the bug
Failure within GeneMark trying to use Funannotate on a fungal genome without any input beyond a FASTA file. Possibly related to #1154?

GeneMark was installed yesterday, downloaded via their website, with the perl headers updated as per their INSTALL file to point at the conda-provided perl in my conda environment:

$  head `which gmes_petap.pl`
#!/mnt/apps/users/pcock/conda/envs/funannotate/bin/perl
# ==============================================================
# Alex Lomsadze
#
# GeneMark-ES Suite version 4.*
#
# Last modified: May, 2021
#
# Please report problems to:
#   Alex Lomsadze    alexl@gatech.edu

What command did you issue?

funannotate predict -i Cc16.pilon-ill.masked.fasta -o . --cpus 1 --species "Cryphonectria carpinicola" --isolate Cc16

This was reproducible after various partial runs on an HPC (with more CPUs assigned), gradually adding missing dependencies like repeat masking and most recently GeneMark.

Logfiles

$ funannotate predict -i Cc16.pilon-ill.masked.fasta -o . --cpus 1 --species "Cryphonectria carpinicola" --isolate Cc16
/mnt/apps/users/pcock/conda/envs/funannotate/lib/python3.11/site-packages/funannotate/funannotate.py:11: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import get_distribution
-------------------------------------------------------
[Jul 30 10:07 AM]: OS: Debian GNU/Linux 12, 16 cores, ~ 49 GB RAM. Python: 3.11.15
[Jul 30 10:07 AM]: Running funannotate v1.8.17
[Jul 30 10:07 AM]: Skipping CodingQuarry as no --rna_bam passed
[Jul 30 10:07 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     busco
  genemark     selftraining
  glimmerhmm   busco
  snap         busco
[Jul 30 10:07 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
[Jul 30 10:07 AM]: Genome loaded: 118 scaffolds; 42,241,886 bp; 8.36% repeats masked
[Jul 30 10:07 AM]: Existing protein alignments found: ./predict_misc/protein_alignments.gff3
[Jul 30 10:07 AM]: Running GeneMark-ES on assembly
[Jul 30 10:07 AM]: CMD ERROR: /home/pcock/GeneMark/gmes_linux_64/gmes_petap.pl --ES --max_intron 3000 --soft_mask 2000 --cores 1 --sequence genome.query.fasta --fungus

real    0m29.263s
user    0m20.774s
sys     0m3.045s

Attempting to run this step directly;

/home/pcock/GeneMark/gmes_linux_64/gmes_petap.pl --ES --max_intron 3000 --soft_mask 2000 --cores 1 --sequence Cc16.pilon-ill.masked.fasta --fungus
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_1.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_1
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_2.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_2
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_3.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_3
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_4.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_4
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_5.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_5
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_6.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_6
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_7.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_7
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_8.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_8
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_9.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_9
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_10.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_10
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_11.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_11
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_12.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_12
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_13.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_13
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_14.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_14
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_15.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_15
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_16.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_16
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_17.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_17
error, file not found /home/pcock/GeneMark/gmes_linux_64/parse_set.pl: set.out
error on call: /home/pcock/GeneMark/gmes_linux_64/parse_set.pl --section ES_A --cfg  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run.cfg  --v

There is no set.out file. Each of those warnings appears to be a segmentation fault:

/home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_ini/es_ini.mod   -f tr  -d /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/plus_hmm.gff  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/run/ES_A_1/hmmout/dna.fa_17.out  /mnt/shared/scratch/pcock/funannotate_Cc16.pilon-ill/data/training/dna.fa_17
Segmentation fault

The gmhmme3 file does seem to run:

$ which gmhmme3
/home/pcock/GeneMark/gmes_linux_64//gmhmme3

$ gmhmme3
GeneMark.hmm eukaryotic, version 3.68_lic

Usage: gmhmme3 [options] <sequence file>

  required parameters:
    -m <model file>

  optional parameters:
    -o <output file>
    -p write protein translation
    -n write nucleotide sequence
    -b <output file> output statistics of predicted introns
    -d <file name> provide input for GeneMark.hmm plus
    -s <string> sequence tag in GFF output format
    -f <format> output prediction in [lst|gff3|gtf] format; default [lst]
    -k <number> value for soft-mask penalty

developer options:
     -z trace back seqid and position
     -w <number> minimum gene length
     -r report best path probability
     -v verbose

$ file `which gmhmme3`
/home/pcock/GeneMark/gmes_linux_64//gmhmme3: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.18, BuildID[sha1]=2fdd65dd00968a67ee2d25a010a83a1396c4c690, stripped

OS/Install Information

• output of funannotate check --show-versions

$ funannotate check --show-versions
/mnt/apps/users/pcock/conda/envs/funannotate/lib/python3.11/site-packages/funannotate/funannotate.py:11: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import get_distribution
-------------------------------------------------------
Checking dependencies for 1.8.17
-------------------------------------------------------
You are running Python v 3.11.15. Now checking python packages...
biopython: 1.79
goatools: 1.6.4
matplotlib: 3.10.9
natsort: 8.4.0
numpy: 2.4.6
pandas: 3.0.5
psutil: 7.2.2
requests: 2.34.2
scikit-learn: 1.9.0
scipy: 1.17.1
seaborn: 0.13.2
All 11 python packages installed


You are running Perl v b'5.032001'. Now checking perl modules...
Carp: 1.50
Clone: 0.46
DBD::SQLite: 1.78
DBD::mysql: 5.013
DBI: 1.651
DB_File: 1.858
Data::Dumper: 2.183
File::Basename: 2.85
File::Which: 1.24
Getopt::Long: 2.58
Hash::Merge: 0.302
JSON: 4.11
LWP::UserAgent: 6.83
Logger::Simple: 2.0
POSIX: 1.94
Parallel::ForkManager: 2.04
Pod::Usage: 1.69
Scalar::Util::Numeric: 0.40
Storable: 3.15
Text::Soundex: 3.05
Thread::Queue: 3.14
Tie::File: 1.06
URI::Escape: 5.35
YAML: 1.30
local::lib: 2.000029
threads: 2.25
threads::shared: 1.61
All 27 Perl modules installed


Checking Environmental Variables...
$FUNANNOTATE_DB=/mnt/shared/scratch/pcock//funannotate_db/
$PASAHOME=/mnt/apps/users/pcock/conda/envs/funannotate/opt/pasa-2.5.3
$TRINITY_HOME=/mnt/apps/users/pcock/conda/envs/funannotate/bin
$EVM_HOME=/mnt/apps/users/pcock/conda/envs/funannotate/opt/evidencemodeler-1.1.1
$AUGUSTUS_CONFIG_PATH=/mnt/apps/users/pcock/conda/envs/funannotate/config/
$GENEMARK_PATH=/home/pcock/GeneMark/gmes_linux_64/
All 6 environmental variables are set
-------------------------------------------------------
Checking external dependencies...
PASA: 2.5.3
CodingQuarry: 2.0
Trinity: 2.15.2
augustus: 3.5.0
bamtools: bamtools 2.5.3
bedtools: bedtools v2.31.1
blat: BLAT v39x1
diamond: 2.2.4
ete3: 3.1.3
exonerate: exonerate 2.4.0
fasta: 36.3.8g
glimmerhmm: 3.0.4
gmap: 2025-07-31
gmes_petap.pl: 4.72_lic
hisat2: 2.2.2
hmmscan: HMMER 3.4 (Aug 2023)
hmmsearch: HMMER 3.4 (Aug 2023)
java: 25.0.1-internal
kallisto: 0.46.1
mafft: v7.526 (2024/Apr/26)
makeblastdb: makeblastdb 2.5.0+
minimap2: 2.31-r1302
pigz: 2.8
proteinortho: 6.3.6
pslCDnaFilter: no way to determine
salmon: salmon 2.3.4
samtools: samtools 1.23
snap: 2006-07-28
stringtie: 3.0.3
tRNAscan-SE: 2.0.13 (Jul 2026)
tantan: tantan 51
tbl2asn: 25.8
tblastn: tblastn 2.5.0+
trimal: trimAl v1.5.rev1 build[2025-11-25]
trimmomatic: 0.41
        ERROR: emapper.py not installed
        ERROR: signalp not installed

[peterjc]

This also happened on a clean run (after removing the earlier partial output) and on all the other fungal genomes I tried (unpublished assemblies). It feels like a tooling issue rather than a data issue.

Is there a suggested small/modest published assembly you use typically suggest as a test case? That might be worth adding to https://funannotate.readthedocs.io/en/latest/tutorials.html too.