34,396 questions
1
vote
1
answer
94
views
Python subprocess with WSL and bcftools not returning awk output
I am building a Flask-based bioinformatics dashboard on Windows using WSL (Ubuntu). I am trying to execute bcftools commands through Python subprocess and calculate statistics from a VCF file.
The ...
2
votes
3
answers
172
views
force number of digits in awk output (no printf)
the problem is that awk transforms any roundable float into integers: 1.000 -> 1
for example:
cat <<EOF | awk -v CONVFMT=%.06f '{$1=$1*2; print}'
0.000 some stuff
1.000 other different ...
- 329
Best practices
0
votes
4
replies
113
views
Bash: printing lines of file until SPECIAL_LINE (not included)
I came up with this:
awk '{print}/SPECIAL_LINE/{exit}' file.txt | head -n -1
Any other suggestions?
Maybe it could be more elegant.
[UPDATE]
Thank you very much everyone! I'd like to point out my ...
- 335
2
votes
3
answers
309
views
Problems redirecting awk output to a file in R
I have a text file (data.txt)
lala=something
dada=somethingelse
stuffwithoutequalsign
otherstuffwithoutequalsign
I am parsing it in R/Rstudio/Win11 using system call to awk (gawk 5.3.0 installed), ...
- 103
3
votes
6
answers
313
views
Merging the headers then concatenate the records of a huge number of text files
I would like to fuse a huge number of big (~1GiB) text files in VCF (Variant Call) Format. The operation requires to read all the headers first (in order to generate and output a new one) and then ...
- 17.8k
8
votes
7
answers
617
views
Extract selected data from multiple lines of text
Here is the output of my curl command (print headers only):
HTTP/1.1 200 OK
Server: CacheHTTPd v1.0
Date: Thu, 08 Jan 2026 19:21:11 +0000
Content-Type: application/octet-stream
Content-Length: ...
- 81
8
votes
4
answers
227
views
Ansible way to convert stdout lines to a dictionary keeping multilines under the same key
Given the following text:
Param1 : Value1
Param2 : Value2
Param3 : Value3a
: Value3b
: Value3c
Param4 : Value4
I would like to obtain the following Ansible result dict of strings or ...
- 2,380
1
vote
2
answers
133
views
awk to get certain next lines when a pattern is matched
I have an input as the following.
0:4:1 untagged 192.168.20.11 255.255.255.0 0.0.0.0 0 0.0.0.0 3205
- 10 192.168.10.11 255.255.255.0 0.0.0.0 0 0.0.0.0 3205
- ...
Advice
1
vote
5
replies
68
views
Find IP addresses with awk matching two strings
I'm trying to get all the IP addresses with response 404's from our Apache web server log file.
This is what I have:
awk '($7 == "404") {print $1,$5}' /var/log/404.log | sort -n > ip.list
...
- 519
Tooling
1
vote
3
replies
97
views
using persistent-memory gawk how variables can created to be local and issolated from other execution instances?
The idea of Persistent-Memory gawk is fabulous because it improves the performance, size, and clarity of many scripts on static and reference data.
However, I have a significant problem in adopting ...
6
votes
4
answers
260
views
awk command to subtract constant from a column and print results
I was working on a one-liner to subtract a constant value (e.g. 100 in this case) from a specific column using awk. So far I can manage to get to where I can print the last iteration only – which ...
- 713
6
votes
3
answers
305
views
Does the awk standard unambiguously define that `a || b=c` is (or is not) valid?
Most awk implementations I have tried (gawk, mawk, original-awk, bsd awk) allow the form: a || b = c
The exception seems to be busybox which returns a syntax error unless parentheses are used:
$ ...
- 19.1k
6
votes
1
answer
167
views
Is there a way to identify if a GNU AWK script is sourced via `-i` / `@include`?
I am looking for a way to write gawk scripts that can function both as standalone tools and function libraries that can be sourced in other scripts.
The difference is in the BEGIN block that I intend ...
- 185
3
votes
3
answers
193
views
splitting a FASTA file into a new FASTA file based on the top 100 transcripts
Essentially, I have a large FASTA file with over 100,000 transcripts in it, and I want to take the top 50 longest of those. After doing this:
awk -vRS=">" -vORS="\n" -vFS="\...
- 39
1
vote
2
answers
171
views
add special characters "[" in print text using awk
this is a part of my bash script
..
mytstamp=$(date '+%Y-%m-%d %H:%M:%S :: ')
output=$(gawk -v mt="$mytstamp" -f print_errlog.awk errlog.txt)
..
my file: <errlog.txt>
2025-10-11 14:25:...
- 331
