bia

joined 1 year ago
[–] bia@programming.dev 1 points 1 month ago

I'm looking at the memory reported by metrics-server in EKS, as that what I base the container resource scaling on. Maybe the go process is reporting memory in a way that doesn't represent the "actual" usage. But I'm not sure it matters here, unless I can get it to change the reported memory usage.

Please see the heap dump I added for 10000 devices. Reported memory is 1,1 GB.

[–] bia@programming.dev 1 points 1 month ago

heap dump from pprof

 

I'm fairly new to go and I've recently migrated a in-memory cache from node to go for concurrency improvements, but the memory usage difference between the two are huge. I've tried to read up on the map memory model in go but haven't been able to find a reason for the difference. I can't see that I'm doing anything special, so I'm looking for guidance here.

The documents that are stored are around 8 KB in size as a JSON file. In node the memory usage for 50000 documents stored as objects is 1,5 GB, and for go maps it is 10 GB.

To me, this doesn't seem reasonable but I can't find the source of the difference. Could anyone here give their take on this?

[–] bia@programming.dev 7 points 4 months ago

I've only worked in software for about 15 years, so don't have much experience outside of git.

But my first job used Microsoft Team Foundation, and I didn't need any experience to know that user experience sucked. I've also done the "date named zip file" type of version control, which is not ideal.

When I started using git it just made sense to me, have had no major complaints since.

[–] bia@programming.dev 2 points 11 months ago (1 children)

I solved it by using indicies instead, that way I got around the borrow checker.

[–] bia@programming.dev 1 points 11 months ago

I used a separate vector when I first solved it, but wanted to try out this solution as well.

Using indices worked, thank you!

[–] bia@programming.dev 3 points 11 months ago

OK, so I'm thinking...

The error says I can't combine immutable and mutable in the same function. I guess this makes sense, and that it's because I'm not allowed to mutate an object while accessing it.

Is this a correct understanding of the error, or is there anything I can do to use this approach?

[–] bia@programming.dev 2 points 11 months ago (3 children)

Thanks, I tried that. But now I get the error.

cannot borrow cards as mutable because it is also borrowed as immutable

[–] bia@programming.dev 2 points 11 months ago* (last edited 11 months ago)

Thank you. I made the change suggested below to the linked one, but it still fails with the error.

cannot borrow cards as mutable because it is also borrowed as immutable

[–] bia@programming.dev 2 points 11 months ago* (last edited 11 months ago)

No, that's probably wrong.

I've just been trying different combinations of borrowing, but I can't get it to works.

I'm pretty sure it's the cards[id].copies += add that is the cause of my issues.

 

Sorry if this is a stupid question, but I'm struggling with what I think is ownership in rust.

I'm completely new to rust, but have done low level languages before so I know the concept of pointers pretty well. However, this rust ownership I can't quite wrap my head around.

I'm trying to refactor my solution to AoC Day 4 Part 2 using a struct reference instead of a stand-alone vector.

The error I'm getting, and can't figure out is in the process function at line

cards.iter_mut().for_each(|card | {

The error is

cannot borrow cards as mutable more than once at a time second mutable borrow occurs here

There is a lot of parsing in my code, so I stuck that in a spoiler below.

The relevant code is:

#[derive(Debug)]
struct Card {
    id: u32,
    score: u32,
    copies: u32,
}

fn process(input: &str) -> u32 {
    let mut cards: Vec = parse_cards(input);

    cards.iter_mut().for_each(|card| {
        let copy_from = card.id as usize + 1;
        let copy_to: usize = copy_from + card.score as usize - 1;

        if card.score == 0 || copy_from > cards.len() {
            return;
        }

        for card_copy in cards[copy_from - 1..copy_to].iter() {
            let id = card_copy.id as usize - 1;
            let add = cards[card.id as usize - 1].copies;
            cards[id].copies += add;
        }
    });

    return cards.iter().map(|c| c.copies).sum();
}

Other code:

spoiler

fn main() {
    let input = include_str!("./input1.txt");
    let output = process(input);
    dbg!(output);
}

fn parse_cards(input: &str) -> Vec {
    return input.lines().map(|line| parse_line(line)).collect();
}

fn parse_line(line: &str) -> Card {
    let mut card_split = line.split(':');
    let id = card_split
        .next()
        .unwrap()
        .replace("Card", "")
        .trim()
        .parse::()
        .unwrap();

    let mut number_split = card_split.next().unwrap().trim().split('|');

    let winning: Vec = number_split
        .next()
        .unwrap()
        .trim()
        .split_whitespace()
        .map(|nbr| nbr.trim().parse::().unwrap())
        .collect();
    let drawn: Vec = number_split
        .next()
        .unwrap()
        .trim()
        .split_whitespace()
        .map(|nbr| nbr.trim().parse::().unwrap())
        .collect();

    let mut score = 0;

    for nbr in &drawn {
        if winning.contains(&nbr) {
            score = score + 1;
        }
    }

    return Card {
        id,
        score,
        copies: 1,
    };
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn full_test() {
        let result = process(
            "Card 1: 41 48 83 86 17 | 83 86  6 31 17  9 48 53
        Card 2: 13 32 20 16 61 | 61 30 68 82 17 32 24 19
        Card 3:  1 21 53 59 44 | 69 82 63 72 16 21 14  1
        Card 4: 41 92 73 84 69 | 59 84 76 51 58  5 54 83
        Card 5: 87 83 26 28 32 | 88 30 70 12 93 22 82 36
        Card 6: 31 18 13 56 72 | 74 77 10 23 35 67 36 11",
        );
        assert!(result != 0);
        assert_eq!(result, 30);
    }
}

[–] bia@programming.dev 5 points 1 year ago (2 children)

Maybe I'm wrong, but using http could create a MITM vulnerability.

And for me this issue is my browser is setup to block http URLs. It's just not a good look.

[–] bia@programming.dev 9 points 1 year ago (5 children)

Hmmm, hosting a hacker post without an https url... 🤨

view more: next ›