An odyssey into the world of concurrency
As developers, our lives are dull and lack the thrill and adventure we might have
experienced in any other profession. Let us narrate an experience that Ananya experienced a week
ago, in her own words which will thrill us enough that a boring C# program to read data from a
database feels like a battlefront—the taste of sweet victory and the repercussions of such a win
everywhere.
It was a mild sunny evening in the bustling city of Chennai with a long weekend around the corner. Many
city dwellers have already vacated to either their hometowns or nearby weekend getaways. leaving it back
for to the likes us. The early part of any career in the modern competitive life always means nil social life. It
is even more as a developer. An already crowded market space where not only humans; these days but
artificial intelligence agents are also in the lurch for coveted software developer titles. Like any other
developer after a few years, I too did not feel like I was new to the task. I felt top of the pecking order after
half a decade to count as my experience. But like nature, the strange complex world of logic has its own
way of putting us down to our rightful place where the recent battlefront of reading data from an SQL server
made me feel nothing short of a developer booting her career up.
The saga
After an eventless standup call the Architect spells out the roadmap. Like any other company, it had
components of generative AI. The influence of buzzwords in our day-to-day life and such words sneaking
into our humble backyard of standup calls are like hand-to-hand skirmishes. Like the one you would have in
the mess hall with fellow bunkmates after a pointless argument. I was as bored as I could ever be. On top
of that, I was not good at doodling! I wish I could add this as a goal for this year’s appraisal. Anyway,
amongst these thoughts I felt I heard my name. Like a commander calling a volunteer to grease the motor
components of a tank to be battle ready my Architect called me to volunteer for a grunt work. After nearly
half a decade to count as experience, I felt I was not cut for such grunt work. But, the pressure to have a
timesheet code to book hours in the week made me willing to volunteer for the grunt work.
Did I not spell what was the work? My bad, it is so dull that I could not stop myself from calling it grunt work.
Frustration apart – The task was to query a database from a C# application. Nothing fancy about it, right?
The only glimmer of challenge I felt was there in that task was the database to query had file tables. It had
a table. The one where a varbinary column housed a file’s bytes. The roadmap had milestones marked for
the C# application to later expand as an observer which applies a few logic filters to pick files. Such files
are cloned into the foreign estates of cloud storage.
First draw
The dullness within me and my utter neglect of real challenges made me write a console application that
did just this
using Dapper;
using System;
using System.Data.SqlClient;
using System.Collections.Generic;
namespace DapperExample
{
class Program
{
static void Main(string[] args)
{
var connectionString = "Did you think I will leave the key to our kingdom here?";
var query = "SELECT * FROM Files";
using (var connection = new SqlConnection(connectionString))
{
try
{
List<File> files = connection.Query<File>(query);
foreach (var file in files)
{
Console.WriteLine($"File ID: {file.Id}, Name: {file.FileName}, Size: {file.FileSize}");
}
}
catch (Exception ex)
{
Console.WriteLine($"Error: {ex.Message}");
}
}
}
Recommended by LinkedIn
}
public class File
{
public int Id { get; set; }
public string Name { get; set; }
public long Size { get; set; }
public byte[] Content { get; set; }
}
}
I carry a personal dislike for the word Information Technology for our profession. If I had my way, I would
happily write it Software * everywhere than spelling Information Technology. This sunny evening, I was
reminded of this personal dislike again. In the profession of Information Technology, information always
flows late, comes out of shape and often never arrives! I was about to send the mail to my boss calling, job
done let me have some social life. The azure-coloured rounded rectangle with the smiling face of my boss
pops up at the right bottom corner of my monitor. That rectangle bore the prodigal information – The keys
to the kingdom. The connection strings.
The perennial; twist
My neglect of real challenges came to bite me. Not that I was unprepared. It did come as a surprise
particularly when the mail claiming it to be a dusted task was in the draft. I learnt that the database was
trailing the production by T-5 days; had 67,000 records in the Files table. Now if we had to lay the
foundations of the ambitious goals of the roadmap, we would have to think of a different strategy. What
happens if the data grows from 67,000 to say 80,000? Are we going to call back the release and make a
new one for say 81,344? Then what happens when the Files count grows to say 1,00,000? Here is where I
believe the real challenge rested. Probably this is why my boss asked me to take a stab at this.
Let me spell the risk. As the rows grow it is quite natural that the memory occupied by the File when the
program runs will grow. Since we are loading it all in a List. The memory occupied by such objects will not
be ready for collection by the vultures of CLR world – Garbage collector. The growth rate is also variable
here if you notice. The File metadata like its Name, Id and Size are relatively small but it is possible that
Content which is a byte array could grow exponentially say within a File count of 100. Thus, making it
impossible to parse even the 67,000 that we know as stands today for volume let alone the lakhs for the
future.
Second draw
I tapped into the world of async programming. Not that I am opening a fantasy land to you. It is nothing new
to you. However, the application of a solution, the real gravity of the solution to a potent problem is the real
thrill we developers thrive upon. So, here is my stab still not final but a shot more worthy than the first code-
and-forget piece.
…
public static async Task Main(string[] args)
{
await foreach (var fileRecord in StreamFileRecordsAsync())
{
Console.WriteLine($"File ID: {fileRecord.Id}, Name: {fileRecord.Name}");
}
}
public static async IAsyncEnumerable<File> StreamFileRecordsAsync()
{
using (var connection = new SqlConnection(ConnectionString))
{
await connection.OpenAsync();
var query = "SELECT * FROM Files";
file = await connection.QueryAsync<File>(sql, commandType: CommandType.Text, buffered:
false));
yield return file;
}
…
Yey! We are now streaming data. The big change is the use of async…await keywords, the option of
buffered: false to the QueryAsync method and then the use of yield with foreach waiting for the whole
operation to finish.
The flip side you ask? I will be a novice to say there is none. For starters what about connection timeouts?
Aha ha, got you. You did not see this coming, right? We will park ourselves in this nice spot of the mystery
of what I did next to jump over that yet un-seen challenge for the next dispatch.
🚀We're Hiring! Join Us at FreshWave Technologies ,USA! 🚀 Ready to take your career to the next level? We're looking for talented individuals to join our dynamic team. Apply now and be a part of our innovative journey! Current Openings - Backend Developer, UI/UX Designer, Project Manager, Software Developer, Data Scientist, Quality Assurance Engineer, DevOps Engineer, Technical Support Specialist, Business Analyst, Digital Marketing Specialist, Sales Executive mail us at hr.freshwavetechnologies@gmail.com #FreshWaveTechnologies #WeAreHiring #JoinUs #TechCareers #ApplyNow
Specialist- Operational Effectiveness & Controls - Review Effectiveness at Standard Chartered Bank
1moWrote 2 emails, tried to reachout to you guys...please try to contact me iam looking for job 0opportunites